The 

Autonomous  Systems  La 

University  of  Hawaii  at  Manoa 


boraf or 


y 


2540  Dole  Street,  Holmes  302 
Dept,  of  Mechanical  Engineering 
University  of  Hawaii 
Honolulu,  Hawaii  96822,  USA 
www.eng.hawaii.edu/~asl 


Development  of  a  Semi-Autonomous 
Underwater  Vehicle  for  Intervention  Missions 

(SAUVIM) 


ONR  Grant  N00014-07-1-0188 
SAUVIM  Phase  lll-C 

10/25/2006  to  10/30/2009 

Final  Technical  Report 


January  28,  2010 

©  2010 


Form  Approved 
OMB  No.  0704-0188 


Report  Documentation  Page 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

JAN  2010 

2.  REPORT  TYPE 

N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

Development  of  a  Semi-Autonomous  Underwater  Vehicle  for  Intervention 

fC  A  TTI7T1\/T  TTT 

5b.  GRANT  NUMBER 

tmssiuna  iruaoc  ni-V/) 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS (ES) 

University  of  Hawaii  at  Manoa  Department  of  Mechanical  Engineering 
2540  Dole  St.,  Holmes  302,  Honolulu,  Hawaii  96822 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS  (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited. 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

SAUVIM  (Semi  Autonomous  Underwater  Vehicle  for  Intervention  Missions)  involves  the  design  and 
fabrication  of  an  underwater  vehicle  that  it  is  capable  of  autonomous  interventions  on  the  subsea 
installations,  a  task  usually  carried  out  by  ROVs  or  human  divers. 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

SAR 

18.  NUMBER 
OF  PAGES 

105 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Copyright : 


Copyright  ©2010  University  of  Hawaii,  Department  of  Mechanical 
Engineering,  the  Autonomous  Systems  Laboratory. 

Permission  to  reprint  or  electronically  reproduce  any  document  or 
graphic  in  part  or  in  its  entirety  for  any  reason  is  expressly 
prohibited,  unless  prior  written  consent  is  obtained  from  the 
University  of  Hawaii,  Department  of  Mechanical  Engineering,  the 
Autonomous  Systems  Laboratory  and  the  proper  entities." 


Summary 

Technical  Development  Report:  October  25,  2006  -  October  30,  2009 


The  SAUVIM  proposal  was  initially  submitted  under  the  ONR  Annual  Announcement  of 
the  July  11,  1996  Commerce  Business  Daily,  and  the  project  officially  began  on  August  1, 
1997  with  its  firs  18-month,  $2,237  million  research  fund  from  the  Office  of  Naval  Research's 
Undersea  Weapons  Technology  Program.  The  project  was  successively  organized  into  six 
further  phases,  from  Phase  II-A  till  the  conclusive  Phase  III-C,  which  is  the  topic  of  the 
current  report. 

Phase  I  of  the  SAUVIM  project  (ONR  GRANT  N00014-97-1-0961)  officially  began  on  August 
1,  1997  with  $2,237  million  from  the  Office  of  Naval  Research's  Undersea  Weapons 
Technology  Program  directed  by  Mr.  James  Fein.  Additional  funding  of  $1,445,000  for  Phase 
II-A  (the  first  part  of  Phase  II)  was  received  on  May  1,  2000  (ONR  GRANT  N00014-00-1- 
0629).  The  second  part  of  Phase  II  ( Phase  II-B )  fund  of  $817,000  was  received  on  June  17,  2002 
(ONR  GRANT  N00014-02-1-0840).  The  third  part  of  Phase  II  (Phase  II-C )  fund  of  $630,000 
was  received  on  August  1,  2003  (ONR  GRANT  N00014-03-1-0969).  Phase  III  ( Phase  III- A) 
fund  of  $480,000  was  received  on  October  1,  2004  (ONR  GRANT  N00014-04-1-0751,  A0001). 
The  second  part  of  Phase  III  (Phase  III-B )  fund  of  $529,950  was  received  on  December  15, 
2005  (ONR  GRANT  N00014-04-1-0751,  A0002).  The  Phase  III-B  has  been  extended  at  no  cost 
until  December  20,  2008.  The  conclusive  third  part  of  Phase  III  (Phase  III-C )  of  $421,048  was 
received  on  October  25,  2006  and  was  extended  at  no  cost  until  October  30,  2009.  Table  1 
summarizes  the  timeline  and  amounts  of  the  SAUVIM  grants  until  December  20,  2008.  See 
also  the  simplified  Gantt  chart  of  Fig.  D. 

In  1999,  with  the  departure  of  Mr.  James  Fein  from  ONR,  Mr.  Chris  Hillenbrand  became  the 
ONR  Program  Officer  for  the  SAUVIM  project.  In  2002,  Dr.  David  Drumheller  became  the 
new  ONR  Program  Officer  for  the  SAUVIM  project.  The  Advisory  Committee  (AdCom)  was 
formed  to  provide  technical  advice  and  direction  by  reviewing  research  directions  and 
progress,  and  to  provide  advice  and  assistance  in  exploring  potential  applications  and  users. 
The  six-member  AdCom  consists  of  Mr.  Fred  Cancilliere  of  Aquidneck  Management 
Associates,  Ltd  (the  former  program  director  of  the  Naval  Undersea  Warfare  Center),  Dr. 
Alexander  Malahoff  of  the  University  of  Hawaii,  Dr.  Homayoun  Seraji  of  the  Jet  Propulsion 
Laboratory,  Dr.  Paul  Yuen  of  the  University  of  Hawaii,  and  Mr.  James  Fein  (the  former  ONR 
Program  Officer  for  SAUVIM)  of  Carderock  Division,  Naval  Sea  Systems  Command.  Mr. 
Dick  Turlington  of  the  Pacific  Missile  Range  Facility  has  retired  and  will  be  replaced  by  Mr. 
Clifton  Ching. 

The  first  progress  report  was  submitted  to  ONR  during  Mr.  Fein's  site  visit  on  October  28, 
1997.  The  second  progress  report  was  submitted  to  ONR  during  the  AdCom's  site  visit  on 
February  24-25, 1998.  The  First  Annual  Report  covering  1997-1998  was  submitted  to  ONR  in 
August  1998  and  presented  during  the  site  visit  on  September  15-16,  1998.  The  fourth 
progress  report  was  submitted  during  Mr.  Hillenbrand's  site  visit  on  April  8,  1999.  The 
Second  Annual  Report  describing  the  overall  technical  progress  of  the  project  during  the 
1998-1999  year  was  submitted  in  July  1999.  The  next  two  ONR  and  AdCom  site  visits  were 
on  May  11,  2000  and  November  14,  2000.  A  Final  Report  for  Phase  I  was  submitted  to  ONR 


in  October  2000.  During  the  next  four  ONR  and  AdCom  site  visits  on  October  29,  2001,  July 
18,  2002,  February  18-19,  2003,  and  October  6,  2003,  initial  balancing  and  motion  wet-tests  of 
SAUVIM  were  conducted,  including  various  surge,  heave,  sway  and  yaw  motions  in  the 
ROV  mode  for  safety  precautions.  On  May  27-28,  2004,  SAUVIM  performed  underwater 
manipulation  tasks  and  simple  navigation  motions  in  the  pier  area.  These  initial 
development  results  were  publicly  shown  on  October  21,  2004  for  80+  attendees  of  the 
Undersea  Defense  Technology  (UDT)  conference.  A  Phase  II-B  Final  Report  was  submitted 
on  March  31,  2005. 

The  next  site  visit  was  on  July  14-15,  2005  where  MARIS  underwater  manipulator 
demonstrated  visual  tracking  performance  in  the  water  with  a  chess  board  image  for  target. 
Successively,  during  the  review  of  April  27  and  28,  2006,  various  tasks  were  performed, 
including  sea  floor  mapping,  vehicle  landing  on  an  underwater  platform,  autonomous 
object  recognition  with  a  camera  on  the  robot  manipulator,  and  autonomous  manipulation 
for  target  retrieval. 

The  successive  Phase  III-B  of  SAUVIM  has  seen  several  major  upgrades  of  the  vehicle, 
including  a  new  power  source  for  enhanced  autonomy,  a  new  wireless  communication  link, 
the  introduction  of  an  Inertial  Navigation  system  and  a  totally  re-designed  navigation 
controller  with  6DOF  performances.  Its  associated  site  visit,  done  on  May  22-23  2007, 
presented  the  whole  set  of  the  new  SAUVIM  upgrades  to  the  AdCom  members. 


Start 

End 

Amount 

Grant  No. 

Phase 

08/01/1997 

10/31/2000 

$  2,237,000.00 

N00014-97-1-0961 

SAUVIM  Phase  1 

05/01/2000 

12/31/2002 

$  1,444,993.83 

N00014-00-1-0629 

SAUVIM  Phase  ll-A 

06/17/2002 

06/30/2004 

$  817,000.00 

N00014-02-1-0840 

SAUVIM  Phase  ll-B 

08/11/2003 

06/30/2006 

$  630,000.00 

N00014-03-1-0969 

SAUVIM  Phase  ll-C 

10/01/2004 

06/30/2006 

$  480,000.00 

N00014-04-1-0751 

SAUVIM  Phase  lll-A 

12/15/2005 

12/20/2008 

$  529,950.00 

N00014-04-1-0751 

SAUVIM  Phase  lll-B 

10/25/2006 

10/30/2009 

$  421,048.00 

N00014-07-1-0188 

SAUVIM  Phase  lll-C 

TOTAL: 

$  6,559,991.83 

Table  1:  SAUVIM  Grants 


The  present  final  report  covers  the  Phase  III-C  of  SAUVIM.  This  is  the  conclusive  phase  of 
the  project,  which  hosted  further  major  upgrades  and,  most  important,  the  demonstration  of 
the  first  fully  autonomous  underwater  manipulation  in  an  unstructured  environment. 
Submerged  in  the  water,  in  its  final  demonstration,  SAUVIM  first  performed  the  self¬ 
calibration  routine,  initializing  its  sub-systems.  After  the  calibration  step,  SAUVIM  began  its 
pre-given  mission—  to  search  for  and  tag  an  underwater  object.  The  object's  location  was 
roughly  given.  Once  the  vehicle  reached  the  area  surrounding  the  object,  it  started  scanning 
the  area  using  a  DIDSON  camera  to  locate  and  identify  the  target.  Once  the  object  was 
detected,  the  vehicle  approached  it  and  positioned  itself  for  optimized  manipulation.  Then, 
while  the  vehicle  was  floating  in  the  water  column,  using  the  unified  coordinated  motion 
control  of  the  vehicle  and  manipulator  system,  the  vehicle  performed  an  autonomous 
manipulation  task  by  applying  a  device  to  the  object  for  tagging. 
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After  completing  the  mission,  the  vehicle  came  back  to  the  dock  by  using  feature-based 
navigation.  The  whole  sequence  was  autonomously  done  and  the  same  mission  was 
successfully  repeated  four  times. 

This  demonstration  presented  a  technological  breakthrough  in  the  field  as  autonomous 
manipulation  had  been  a  bottleneck  issue  for  underwater  intervention  missions. 


Objective 

Many  underwater  intervention  tasks  are  today  performed  using  manned  submersibles  or 
Remotely  Operated  Vehicles  in  tele-operation  mode.  Autonomous  Underwater  Vehicles  are 
mostly  employed  in  survey  applications.  In  fact,  the  low  bandwidth  and  significant  time 
delay  inherent  in  acoustic  subsea  communications  represent  a  considerable  obstacle  to 
remotely  operate  a  manipulation  system,  making  it  impossible  for  remote  controllers  to 
react  to  problems  in  a  timely  manner. 

Nevertheless,  vehicles  with  no  physical  link  and  with  no  human  occupants  permit 
intervention  in  dangerous  areas,  such  as  deep  ocean,  under  ice,  in  missions  to  retrieve 
hazardous  objects,  or  in  classified  areas.  The  key  element  in  underwater  intervention 
performed  with  autonomous  vehicles  is  autonomous  manipulation,  which  refers  to  the 
capability  of  a  robot  system  that  performs  intervention  tasks  requiring  physical  contacts 
with  unstructured  environments  without  continuous  human  supervision. 

This  challenging  technology  milestone  has  been  our  long-term  objective,  through  the 
development  of  a  Semi- Autonomous  Underwater  Vehicle  for  Intervention  Missions 
(SAUVIM):  an  undersea  robot  that  can  intelligently  work  with  arms  rather  than  just  swim , 
significantly  advancing  the  Navy's  ability  for  undersea  intervention  missions. 

Today  only  few  AUVs  are  equipped  with  manipulators.  SAUVIM,  at  its  current  state  of  the 
art,  is  one  of  the  first  underwater  vehicles  designed  to  perform  autonomous  manipulation 
tasks. 

The  SAUVIM  technical  approach  to  underwater  intervention  involved  the  development  of  a 
robust  autonomous  manipulation  framework  over  several  years  of  researches. 

Our  current  results  represent  an  important  passage  toward  the  development  of  a  higher 
level  of  autonomy  for  intervention  AUVs,  providing  a  cost-effective  engineering  solution  to 
many  new  underwater  tasks  and  applications  that  the  fly-by  type  submersibles  have  not 
been  able  to  handle. 


Program  Implementation 

During  the  overall  phases,  research  for  SAUVIM  was  carried  out  by  continued  coordination 
of  three  organizations:  Autonomous  Systems  Laboratory  (ASL)  of  the  University  of  Hawaii 
(UH),  Marine  Autonomous  System  Engineering,  Inc.  (MASE),  and  Naval  Undersea  Warfare 
Center ,  Newport  (NUWC). 
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Junku  Yuh  has  been  the  PI  of  the  SAUVIM  project  from  the  ASL  organization  for  Phase  I, 
Phase  II  and  Phase  III-A.  He  continued  to  serve  as  PI  during  the  Phase  III-B  of  SAUVIM,  for 
ASL. 

Frederick  Cancilliere  was  the  PI  of  the  SAUVIM  project  from  the  NUWC  organization  for 
Phase  II-B.  Due  to  Mr.  Cancilliere' s  retirement,  Paul  Temple  has  served  as  PI  for  Phase  II-C 
and  Phase  III-A  for  NUWC.  He  continued  to  serve  as  the  PI  for  NUWC  for  Phase  III-B. 

Song  K.  Choi  has  been  the  PI  of  the  SAUVIM  project  from  the  MASE  organization  for  Phase 
II,  Phase  III-A  and  III-B.  He  will  continue  to  serve  as  the  PI  for  Phase  III-C  for  MASE. 

The  ASL  is  the  primary  research  organization  for  the  SAUVIM  project.  ASL  staff  members 
have  developed  key  technologies  of  the  SAUVIM  project  such  as  vehicle  control  system 
with  real-time  operating  system,  underwater  navigation  algorithm,  sensor  handling 
algorithm,  sensor  fusion,  robot  manipulator  control  system,  and  underwater  image 
processing  system.  The  ASL  at  UH  has  been  used  to  train  highly  capable  engineers  and 
scientist  to  contribute  to  the  underwater  technologies  society  from  various  industries. 

While  Junku  Yuh  has  been  the  official  PI  for  ASL  during  all  the  SAUVIM  Phases,  the 
development  of  the  Phase  III-B  has  been  coordinated  by  Giacomo  Marani  from  ASL,  starting 
from  December  2006.  During  the  same  period,  Giacomo  Marani  served  also  as  SAUVIM 
Acting  PI.  Tae  Won  Kim  was  the  Project  Co-Coordinator  for  Phase  II  and  has  been  the  Co-PI 
from  ASL  and  the  Project  Coordinator  of  the  SAUVIM  project  for  Phase  III-A  and  the 
beginning  of  Phase  III-B,  until  December  2006. 

Giacomo  Marani  contined  serving  as  SAUVIM  acting  Principal  Investigator  during  the  whole 
current  Phase  III-C,  until  the  end  of  the  project,  and  leading  the  main  technical  coordination 
betweent  he  SAUVIM  entities. 

MASE  is  the  spin-off  company  from  the  ASL,  UH.  The  key  MASE  staff  members  are  former 
members  of  ASL,  who  were  involved  in  the  design,  analysis,  fabrication  and  testing  of 
SAUVIM  in  Phase  I  and  Phase  II-A.  Song  K.  Choi  served  as  the  SAUVIM  Program 
Coordinator  during  Phase  I  and  as  the  Associate  Director  in  Phase  II-A.  He  has  been  the  PI 
of  the  SAUVIM  project  from  the  MASE  organization  for  Phase  II  and  Phase  III-A.  MASE's 
contribution  to  the  proposed  research  is  essential  as  MASE  staff  members'  profound 
research  experiences  and  skills  especially  with  SAUVIM  as  well  as  their  private  sector 
environment  are  crucial  factors  to  complete  this  project  with  respect  to  research  outcome  in 
industrial  standards  and  future  technology  transfer.  MASE  plans  to  continuously  provide 
engineering  service  for  maintenance,  modifications,  and  field  operations  of  SAUVIM. 
SAUVIM  is  ultimately  owned  by  UH  and  will  be  used  for  UH  and  Navy  tasks  as  priorities. 

NUWC  is  the  main  Navy  laboratory  where  Navy's  key  projects  in  unmanned  underwater 
vehicles  (UUVs)  have  been  carried  out.  NUWC  possesses  a  great  abundance  of  research  and 
operational  experience  of  UUVs,  especially  with  the  fly-by  type  AUVs.  Mr.  Cancilliere  has 
served  as  a  member  of  the  SAUVIM  Advisory  Committee  and  is  very  familiar  with  the 
research  objectives  and  progress  of  SAUVIM.  Mr.  Cancilliere  initiated  the  maintenance, 
safety,  and  testing  documentation  during  Phase  II-B.  During  Phase  II-C  and  III-A,  Paul 
Temple  has  continued  to  lead  the  NUWC  work  by  utilizing  UNWC  personnel  who  are 
already  familiar  with  the  SAUVIM  project. 

While  their  joint  involvements  are  at  different  levels  in  the  program,  integrated  research 
efforts  of  all  three  organizations  are  essential  for  the  successful  completion  of  the  SAUVIM 
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project.  ASL  is  focused  on  the  theoretical  investigation  and  software  development;  MASE  is 
focused  on  the  experimental  testing,  hardware  development,  and  sensor  and  power 
investigations;  and  NUWC  is  focused  on  the  experimental  implementation  of  the  proposed 
research  tasks,  sea  trials,  and  documentation.  From  December  2006,  until  the  end  of  the 
project,  the  overall  technical  coordination  between  the  project  entities,  particularly  among 
the  research  institute  (ASL)  and  engineering  service  (MASE),  was  managed  by  Giacomo 
Marani. 

The  SAUVIM  revised  organizational  diagram  is  shown  in  Figure  A  and  a  simplified 
SAUVIM  schedule  is  shown  in  Figure  D. 


Figure  A.  SAUVIM  Revised  Organizational  Diagram 
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Background 


It  is  clear  from  various  meetings  with  Navy  experts  and  the  autonomous  underwater 
vehicle  (AUV)  community  that  there  is  a  great  need  for  improving  undersea  intervention 
capabilities  in  terms  of  autonomy,  cost-effectiveness,  and  performance.  Various  underwater 
intervention  tasks  include  underwater  plug/ unplug,  construction  and  repair,  cable 
streaming,  mine  hunting,  and  munitions  retrieval.  All  underwater  vehicles  currently  used 
for  intervention  missions  are  either  manned  submersibles  or  remotely  operated  vehicles 
with  manipulators.  These  vehicle  operations  are  expensive  and  often  face  a  number  of  safety 
issues.  Furthermore,  their  performance  in  terms  of  accuracy  and  efficiency  are  questionable, 
mainly  due  to  human  operator  fatigue  and  the  time  delay  in  the  man-machine  control  loop 
in  an  unstructured  environment.  Even  though  recent  advances  in  sensors,  communication, 
computers,  and  machine  intelligence  have  made  it  possible  to  attempt  to  design  advanced 
AUVs,  the  AUV  development  is  still  mostly  directed  toward  a  survey-oriented  vehicles. 

In  literature  there  are  only  few  examples  of  Intervention  AUVs.  These  example  include  the 
OTTER  I-AUV  by  the  Stanford  Aerospace  Robotics  Lab.  OTTER,  developed  back  in  1996,  is 
a  hover  capable  underwater  vehicle  which  operates  in  a  test  tank  at  the  Monterey  Bay 
Aquarium  Research  Institute  (MBARI).  Current  and  past  research  includes  texture-based 
vision  processing  for  feedback  control  and  real-time  mosaicking,  autonomous  intervention 
missions,  and  hydrodynamic  modeling  of  underwater  manipulators.  A  study  on  automatic 
objects  retrieval  was  done  in  [Wang95]. 

Another  Intervention  AUV,  ALIVE,  was  developed  in  2003  by  Cybernetix.  The  aim  of  the 
EU-funded  ALIVE  project  was  to  develop  an  Intervention-AUV  capable  of  docking  to  a 
subsea  structure  which  has  not  been  specifically  modified  for  AUV  use.  A  description  of  the 
ALIVE  vehicle  was  given  in  [Evans03]. 

The  key  element  in  underwater  intervention  performed  with  autonomous  vehicles  is 
autonomous  manipulation.  This  is  a  challenging  technology  milestone,  which  refers  to  the 
capability  of  a  robot  system  that  performs  intervention  tasks  requiring  physical  contacts 
with  unstructured  environments  without  continuous  human  supervision. 

Intervention  missions  requiring  physical  contact  with  the  surroundings  in  the  unstructured, 
underwater  environment  always  increase  the  level  of  risk  in  damaging  the  system  and 
present  completely  different  dynamic  problems  from  fly-by,  non-contact  type  operation. 
The  overall  motion  of  the  vehicle-manipulator  system  is  a  high  degrees-of-freedom  (dof) 
operation  due  to  additional  dof  of  the  manipulator  added  to  the  vehicle's  six  dof.  Operation 
requires  a  high  degree  of  precision  and  accuracy,  which  accomplishment  if  often 
complicated  by  the  presence  of  an  external  disturbance  such  as  a  current.  All  of  these  issues 
present  very  complex  engineering  problems  that  have  hindered  the  development  of  AUVs 
for  intervention  missions. 

Autonomous  manipulation  systems,  unlike  teleoperated  manipulation  systems,  that  are 
controlled  by  human  operators  with  the  aid  of  visual  and  other  sensory  feedback,  must  be 
capable  of  assessing  a  situation,  including  self-calibration  based  on  sensory  information,  and 
executing  or  revising  a  course  of  manipulating  action  without  continuous  human 
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intervention.  It  is  sensible  to  consider  the  development  of  autonomous  manipulation  as  a 
gradual  passage  from  human  teleoperated  manipulation. 

Within  this  passage,  the  most  noticeable  aspect  is  the  increase  of  the  level  of  information 
exchanged  between  the  system  and  the  human  supervisor. 

In  teleoperation  with  ROVs,  the  user  sends  and  receives  low  level  information  in  order  to 
directly  set  the  position  of  the  manipulator  with  the  aid  of  a  visual  feedback.  As  the  system 
becomes  more  autonomous,  the  user  may  provide  only  a  few  higher  level  decisional 
commands,  interacting  with  the  task  description  layer.  The  management  of  lower  level 
functions  (i.e.  driving  the  motors  to  achieve  a  particular  task)  is  left  to  the  onboard  system. 
The  level  of  autonomy  is  related  to  the  level  of  information  needed  by  the  system  in 
performing  the  particular  intervention.  At  the  task  execution  level,  the  system  must  be 
capable  of  acting  and  reacting  to  the  environment  with  the  extensive  use  of  sensor  data 
processing. 

The  user  may  provide,  instead  of  directly  operating  the  manipulator,  higher  level 
commands  during  a  particular  mission,  such  as  "unplug  the  connector".  In  this  approach, 
the  function  of  the  operator  is  to  decide,  after  an  analysis  of  the  data,  which  particular  task 
the  vehicle  is  ready  to  execute  and  successively  to  send  the  decision  command.  The  low- 
level  control  commands  are  provided  by  a  pre-programmed  onboard  subsystem,  while  the 
virtual  reality  model  in  the  local  zone  uses  only  the  few  symbolic  information  received 
through  the  low  bandwidth  channel  in  order  to  reproduce  the  actual  behavior  of  the  system. 

This  report  presents  the  solutions  chosen  to  address  the  above  issues  for  autonomous 
manipulation,  developed  during  the  course  of  the  SAUVIM  research  project,  and  concluded 
with  the  demonstration  of  the  first  fully  autonomous  underwater  manipulation  in  an 
unstructured  environment. 

The  proposed  study  is  in  response  to  current  local  and  national  needs  for  the  development 
of  this  technology  and  will  ultimately  be  useful  in  many  intervention  missions.  SAUVIM 
technologies  could  be  extended  for  harbor  security  that  would  be  part  of  homeland  security , 
one  of  our  nation's  current  interests  and  concerns.  One  potential  user  is  the  Pacific  Missile 
Ranging  Facility  (PMRF)  of  the  U.S.  Navy  in  the  State  of  Hawaii. 
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Research  summary 


The  SAUVIM  project  was  proposed  as  a  two-phase  research  and  development  program. 
Phase  I  had  three  parts:  (1)  to  study  the  major  research  components,  (2)  to  develop  and 
integrate  the  basic  software  and  hardware  of  SAUVIM,  and  (3)  to  test  the  vehicle  in  a 
shallow  water  environment.  Phase  II  is  a  continuation  and  completion  of  the  research  and 
development  of  Phase  I  with  water  environment  testing. 

As  stated  in  the  original  proposal,  the  project  consists  of  five  major  components: 

•  Adaptive,  Intelligent  Motion  Planning; 

•  Automatic  Object  Ranging  and  Dimensioning; 

•  Intelligent  Coordinated  Motion/Force  Control; 

•  Predictive  Virtual  Environment;  and 

•  SAUVIM  Design. 

During  the  Phase  I  period,  there  have  been  approximately  sixty  people  supported  by  this 
ONR  project.  In  2007,  there  were  7  people  working  on  the  project  in  ASL  -  1  faculty 
members,  3  full-time  staff  members,  2  undergraduate  students,  and  1  administrative 
assistant.  The  Advisory  Committee  was  formed  to  provide  technical  advice  and  direction 
by  reviewing  research  directions  and  progress,  and  to  provide  advice  and  assistance  in 
exploring  potential  applications  and  users.  The  four-member  Advisory  Committee 
consisted  of  Mr.  Fred  Cancilliere  of  the  Aquidneck  Management  Associates,  Ltd.,  Dr. 
Alexander  Malahoff  of  the  University  of  Hawaii,  Dr.  Homayoun  Seraji  of  the  Jet  Propulsion 
Laboratory,  and  Mr.  Dick  Turlington  of  the  Pacific  Missile  Range  Facility.  Two  additional 
members  -  Dr.  Paul  Yuen  of  the  University  of  Hawaii  and  Mr.  James  Fein,  the  former  ONR 
Program  Director  -  have  been  included  in  the  Advisory  Committee. 

•  Adaptive,  Intelligent  Motion  Planning  (AIMP)  -  The  AIMP  aims  at  developing 
SAUVIM' s  motion  planning,  which  is  intelligent  and  adaptive  in  that  the  system  is 
capable  of  decision-making  at  a  task  or  mission  level  and  can  deal  with  unknown  or 
time-varying  environments.  Motion  planning  for  an  AUV  can  be  decomposed  into  path 
planning  and  trajectory  generation,  although  they  are  not  completely  independent  of 
each  other.  Path  planning  is  a  computation  and  optimization  of  a  collision-free  path  in 
an  environment  with  obstacles.  Trajectory  generation  is  the  scheduling  of  movements 
for  an  AUV  along  the  planned  path  over  time.  To  simultaneously  compensate  for  these 
objectives,  a  genetic  algorithm  (GA)  based  3D-motion  planner  was  studied  both  off-line 
and  on-line  cases.  In  general,  and  for  any  algorithms,  an  off-line  case  is  when  an 
environment  is  known  and  static,  while  an  on-line  case  must  be  capable  of  modifications 
in  response  to  dynamic,  environmental  changes.  The  utilization  of  GA-based  approach 
has  two  advantages:  1)  it  is  adaptive  and  2)  the  dimension  of  space  has  less  effect  on 
performance  than  other  methods. 

The  AIMP  software  has  gone  through  three  version  upgrades.  The  first  was  Version 
1. alpha,  which  integrates  the  off-line  and  on-line  algorithms  in  C  with  a  graphic  user 
interface  using  OpenGL.  This  software  version  was  tested  on  the  Autonomous  Systems 
Laboratory's  autonomous  underwater  vehicle  -  ODIN.  The  second  was  Version  1.0, 
which  integrates  the  path  planning  and  trajectory  generation  algorithms.  The  third  was 
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Version  1.1,  which  optimizes  the  original  software  organization  and  data  structures,  and 
includes  a  database  of  mapping  data  on  the  main  memory.  Also,  a  Software 
Development  Process  (SDP)  has  been  developed  and  implemented  to  oversee  the 
various  developments  in  software  version  changes.  Several  papers  have  been  published 
in  these  subjects. 

During  an  attempt  to  make  an  on-line  version  of  AIMP  in  Phase  II-C,  it  was  found  that 
there  was  no  ending  condition  of  genetic  evolution  to  build  a  3-D  motion  planner. 
There  was  no  measure  to  guarantee  optimality  of  the  generated  3-D  path  or  trajectory  as 
well.  Thus  a  conventional  math-based  motion  planning  method  is  implemented  in 
Phase  III- A,  and  a  new  motion  planning  algorithm  has  been  investigated  for  complex 
motion  in  3-D  as  well  as  minimizing  computing  burden. 

Phase  III-B  has  seen  an  increase  in  the  degrees  of  freedom  that  the  motion  planner  is  able 
to  handle,  in  order  to  better  cope  with  the  requirements  of  an  underwater  intervention. 
The  conventional  math-based  motion  has  been  extended  in  order  to  optimize  the 
rotational  and  translational  movements  of  the  vehicle,  allowing  precise  positioning  of 
the  robotic  arm  in  the  area  of  interest.  This  is  usually  different  from  the  fly-by  type  AUV 
where  the  primary  goal  is  to  survey  a  generic  area. 

•  Automatic  Object  Ranging  and  Dimensioning  (AORD)  -  The  main  objective  of  the 
AORD  is  to  develop  a  multiple  sensor  system  to  be  utilized  during  SAU  VIM's 
intervention  missions  to  locate  the  target.  This  system  originally  consisted  of  three- 
sensors: 

1.  Laser  ranging  sensor  (LRS), 

2.  Passive  arm  sensor  (PA) 

3.  Manipulator  homing  sensor  (MHS) 

The  laser  ranger,  the  homing  sensor,  and  the  passive  arm  have  all  been  designed  and 
prototyped  in  the  previous  phases. 

The  underwater  prototypes  of  the  LRS  has  been  fabricated,  assembled  and  tested,  with 
the  camera  housings  manufactured  using  6061  aluminum  with  vacuum-sealed  lens.  The 
software  has  been  developed  using  the  prototypes. 

The  PA,  in  its  original  configuration,  was  made  of  6061-Aluminum,  and  it  had  two 
three-axis  gimbaled  joints  and  a  single-axis  hinge  joint.  The  entire  PA  structure  was 
compensated  with  mineral  oil.  It  utilized  the  original  software  developed  for  the 
prototype.  The  kinematics  of  the  PA  has  been  re- verified  using  various  symbolic  math 
packages.  The  passive  arm  has  also  been  rewired  for  optimal  performance.  It  was 
simulated  with  the  active  arm  to  conduct  feasibility  studies  in  obtaining  active 
manipulation  position.  However,  after  a  long  investigation,  it  was  concluded  that  the  PA 
cannot  be  easily  deployed  and  retrieved  in  the  water  due  to  the  lack  of  active  power  in 
the  arm.  Thus,  an  underwater  version  of  the  ultrasound  motion  tracker  has  been 
introduced  as  replacement  of  the  passive  arm  system.  Since  there  are  no  commercial 
versions  of  similar  devices,  a  prototype  version  of  the  ultrasonic  motion  tracker  had  to 
be  developed  in  house. 

The  original  idea  of  the  homing  sensor  was  to  use  a  dedicated  PC104  computer  with 
camera  to  detect  a  simple  circular  barcode.  It  was  originally  tested  to  confirm  its 
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performance  in  the  water,  and,  despite  results  were  good  enough  to  use  the  bar  code  in 
the  water,  it  suffered  of  obvious  application  limitations.  During  the  past  years,  and 
especially  starting  with  the  Phase  III-B,  the  idea  shifted  toward  a  more  organized  and 
range  dependant  Target  Identification  procedure. 

The  localization  subsystem  has  been  subject  of  major  upgraded  during  the  Phase  III-B, 
and  finalized  to  a  fully-working  and  extensively  tested  subsystem  during  the  last  Phase 
III-C.  The  target  localization  process,  which  is  the  main  support  for  the  capabilities  of  the 
autonomous  manipulation  of  SAUVIM,  in  its  final  configuration,  is  performed  by  using 
and  fusing  different  technologies  (acoustical  and  optical)  in  order  to  guarantee  a 
suitable,  range  dependent,  level  of  reliability,  precision  and  accuracy.  The  SAUVIM 
AUV  switches  through  three  main  sensing  methods  in  order  to  acquire  reliable  data.  As 
shown  in  Figure  B,  the  sensor  technology  changes  according  to  the  combination  of  range 
and  accuracy  needed. 

In  long  range  (over  25m),  375KHz  image  sonars  are  used  for  initial  object  searching.  The 
accuracy  in  this  range  is  necessary  only  to  direct  the  vehicle  toward  the  target  zone. 


Target 

Short  range: 

Camera,  Motion 
Tracker 

Medium  range: 

DIDSON 

Long  range: 

Sidescan  sonar, 
Imaging  sonar 


Figure  B.  The  phases  involved  in  a  search  for  the  target. 

In  mid-range  (2-25m),  a  Dual  frequency  IDentification  SONar  (DIDSON)  is  used  for 
object  recognition  and  the  vehicle  positioning.  This  is  the  phase  where  the  vehicle  has  to 
position  itself  in  order  to  have  the  target  confined  within  the  manipulation  workspace. 
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At  this  range,  and  in  case  of  turbid  water,  it  is  virtually  impossible  to  use  conventional 
optical  cameras  to  identify  an  object.  This  justifies  the  use  of  the  DIDSON,  which  has 
been  used  as  a  ranging  sensor  from  Phase  II-C  onwards.  During  the  Phase  III-B  our 
focus  has  been  directed  toward  refining  the  algorithms  of  autonomous  target 
identification  with  the  DIDSON.  In  the  final  Phase  III-C  the  algorithms  have  been 
implemented  and  successfully  tested  in  repeatability  and  robustness,  thus  allowing  the 
SAUVIM  vehicle  to  reliably  find  a  path  toward  the  target  area. 

Finally,  when  the  target  is  within  the  manipulator  workspace,  short  range  and  high 
accuracy  sensor  are  used  in  order  to  perform  the  actual  intervention  task.  This  goal  is 
achieved  with  the  combined  use  of  underwater  video  cameras  and  the  ultrasonic  motion 
tracker  described  above,  used  to  retrieve  the  real-time  6  DOF  position  of  the  target 
during  the  manipulation  tasks.  The  device  utilizes  high  frequency  sound  waves  to  track 
a  target  array  of  ultrasonic  receivers.  The  use  of  4  transmitters  at  the  stationary  positions 
with  4  receivers  on  the  target  can  be  used  to  determine  the  6  DOF  generalized  position 
(rotation  and  translation)  of  the  object. 


•  Intelligent  Coordinated  Motion/ Force  Control  (ICM/FC)  -  The  major  objective  of  the 
ICM/FC  is  simple  yet  complex.  The  control  of  an  AUV  and  its  manipulator  is  a  multi¬ 
bodied,  dynamic  problem  of  vast  unknowns;  therefore,  this  task  was  subdivided  into 
four  sub-tasks,  which  were: 

Theoretical  Modeling  (TM) 

Low-Level  Control  (LLC) 

High-Level  Control  (HLC) 

Dry  Test  Design  and  Set-up  (DTDS). 

However,  with  the  arrival  of  the  7-dof  underwater  manipulator,  the  TM  and  DTDS  were 
combined  to  form  a  common  group  -  Manipulator  Control  and  Test  Platform  (MCTP). 
Also,  a  Localization  and  Navigation  (LN)  group  was  spun-off  the  LLC  group  due  to  the 
vastness  and  complexity  of  the  LN  material.  The  LN  group  was  trying  to  devise  a 
hybrid  localization  and  navigation  methodology  that  will  suffice  in  understanding  the 
geophysical,  terrain-matching  and  dead-reckoning  aspects  for  proper  navigation.  An 
integrated  data  fusion  methodology  was  also  being  devised  to  quickly  and  correctly 
digest  the  immense  amounts  of  data  from  the  sensors,  which  undoubtedly  has  mass 
abundance  of  noise  and  errors.  However,  it  was  found  that  the  map-based  localization 
method  is  a  task  computationally  intense  and,  despite  this  aspect,  does  not  meet  the 
accuracy  requirement  for  the  vehicle  control.  Thus,  a  Ultra  Short  Base  Line  (USBL) 
device  has  been  used  as  a  vehicle  monitoring  sensor  as  well  as  a  position  feedback 
sensor. 

The  MCTP  was  developed  to  accelerate  the  progress  in  the  TM  and  DTDS  sub-tasks. 
With  the  acquirement  of  the  MARIS  7080  manipulator  and  constraints  in  time,  the  focus 
has  been  changed  to  the  development  of  the  arm  software  in  conjunction  with  the 
manipulator  kinematics,  dynamics,  force-control  and  coordinated  motion  control 
modules.  During  the  Phase  II  of  SAUVIM  the  Maris  7080  manipulator  initially  ran  off 
the  VME  bus  system  using  VxWorks  and  Matlab  with  Simulink.  Development  in  the 
"rapid  prototyping,  graphic  software"  has  been  the  central  point  in  enhancing  the 
complex,  underwater  dynamic  actions  and  reactions.  The  manipulator  control  code  has 
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been  developed  to  perform  force/ torque  tasks,  path  optimization  around  singularity 
points,  and  collision  avoidance  techniques.  Successively  the  development  approach 
moved  from  the  rapid  prototyping  mode  to  the  stand-alone  mode,  in  order  to  optimize 
the  performances  in  the  vehicle. 

In  phase  II-B,  a  new  parking  procedure  was  developed  and  tested  in  the  water.  This  is 
one  of  the  most  critical  tasks  of  the  manipulation  system,  due  to  the  limited  space  for  the 
manipulator  in  the  vehicle.  The  Arm  Programming  Language  (APL)  was  developed  for 
high  level  control  of  the  manipulator  without  changing  system  S/W,  and  an  ultrasound 
motion  tracker  was  interfaced  (in  the  air)  to  the  manipulator  to  get  precise  position 
feedback  information,  used  either  for  calibration  purposes  and  for  initial  dry  tests  of 
target  tracking.  The  underwater  version  of  the  motion  tracker  is  under  development  for 
substituting  the  passive  arm  which  was  originally  planned  to  measure  relative 
position/ orientation  between  the  target  object  and  the  vehicle.  Preliminary  results 
obtained  during  the  Phase  II-B  showed  a  very  high  precision  in  position  measurement. 

In  Phase  III- A,  image  processing  module  in  robot  system  was  upgraded  including  new 
frame  grabber  and  image  processing  library  for  Intel  CPU.  The  phase  III-B  has  seen 
further  improvements  of  the  camera  system,  with  added  procedure  for  auto-calibration 
to  be  performed  directly  on  the  target  site  (underwater)  in  order  to  compensate  for  the 
local  water  condition. 

The  Phase  III-C  brought  to  the  manipulation  subsystem  further  upgrades  and  extensive 
testing  of  the  system:  a  more  accurate  object  detection  capability,  a  workspace 
optimization  controller  and  a  reliable  collision  avoidance  system.  The  final 
demonstration  confirmed  great  repeatability  and  robustness  of  the  robotic  subsystem. 

The  LLC  was  created  with  two  objectives:  1)  to  design  and  develop  an  advanced  vehicle 
control  system  for  navigation  and  hovering,  and  2)  to  design  and  develop  an  advanced 
coordinate  motion/ force  control  system  of  the  vehicle  and  manipulator  during  the 
intervention  mode.  However,  with  the  creation  of  the  LN  group,  the  emphasis  was  on 
the  integration  of  the  localization  and  navigation  techniques  to  the  basic  motion  and 
hovering  tasks.  During  the  Phase  III-A  the  development  of  the  coordinated 
motion/force  control  system  was  being  explored  from  two  separate  platforms.  As  the 
MCTP  development  continued,  the  LLC  was  optimizing  the  hovering  and  station¬ 
keeping  methodologies  on  the  ODIN  vehicle.  Various  types  of  modern  controllers,  such 
as  Adaptive  controller.  Disturbance  Observer  (DOB)  controller.  Adaptive  DOB 
controller,  and  Neuro-fuzzy  controller,  were  investigated  in  order  to  find  the  best 
controller  for  the  underwater  vehicle. 

In  all  the  SAUVIM  Phases,  the  focus  of  the  LN  group  has  been  on  efforts  in  obtaining 
high  performance  in  navigation  and  hovering.  Before  the  current  phase  III-B  the 
navigation  and  hovering  techniques  made  use  of  the  data  from  the  on-board  scan  sonar, 
altimeter  sonar,  inertial  navigation  unit,  Doppler  Velocity  Logger  (DVL),  and  pressure 
sensors.  The  Ultra  Short  Base  Line  (USBL)  and  Global  Positioning  System  (GPS)  were 
added  as  a  global  vehicle  position  feedback  sensor  during  underwater  navigation  and 
surface  navigation,  respectively. 
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However,  at  the  end  of  the  Phase  III- A,  it  was  evident  how  the  accuracy  and  precision  of 
this  sensor  system  was  insufficient,  in  particular  conditions,  during  the  manipulation 
tasks.  Thus  it  was  necessary,  during  the  phase  III-B,  to  introduce  a  more  reliable  Inertial 
Navigation  System  aimed  to  produce  the  position  data  with  the  reliability  necessary  to 
the  autonomous  intervention. 

This  important  change,  together  with  a  complete  re-design  of  the  navigation  controller, 
allowed  the  SAUVIM  vehicle  to  successfully  perform  in  compliance  with  the  precision, 
accuracy  and  stability  requirements  of  our  manipulation  task. 

Another  important  upgrade  of  the  Phase  III-B,  aimed  to  improve  the  coordinate  motion 
between  the  arm  and  the  vehicle,  was  the  standardization  of  all  the  communication 
protocols.  This  was  accomplished  with  the  extension  of  the  xBus  protocol,  once  dedicate 
to  the  arm  subsystem  only,  to  the  entire  vehicle.  xBus  showed  a  great  flexibility  in 
handling  every  kind  of  communication  (data,  program  code,  messages..  )  and  thus  it 
was  chosen  as  the  SAUVIM  standard. 

xBus  uses  a  client-server  approach  for  delivering  information  from  and  to  each 
distributed  module.  Each  subsystem  (as  a  backset  module  or  a  generic  sensor)  embeds  a 
custom  TCP-IP  client-server  communication  system  (see  [Marani05]).  Within  this 
architecture,  every  server  can  deliver  the  requested  information  on-demand  to  any 
number  of  clients,  and  this  configuration  allows  a  different  utilization  of  the  bandwidth, 
since  every  data  is  broadcasted  only  on  demand. 

This  approach  is  similar  to  the  Publish-Subscribe  Middleware  paradigm  [Ben07],  where 
the  term  "middleware"  refers  to  the  architecture  software  that  coordinates  the  set  of 
software  modules  collectively  comprising  the  backseat-driver  system  running  in  the 
payload.  Publish-subscribe  middleware  implements  a  community  of  modules 
communicating  through  a  shared  database  process  that  accepts  information  voluntarily 
published  by  any  other  connected  process  and  distributes  particular  information  to  any 
such  process  that  subscribes  for  updates  to  such  information. 

In  the  SAUVIM  approach  the  information  is  not  published  by  a  central  database,  but 
every  source  acts  as  a  server  that  may  send  only  the  requested  information  to  the 
requesting  client.  The  distributed  client-server  architecture  also  provides  a  security 
hand-shaking  mechanism,  which  provides  direct  feedback  on  the  execution  of  any 
instance  of  data  exchange.  This  is  particularly  desirable  in  issuing  security  commands 
(such  as  for  aborting  the  mission). 

During  the  Phase  III-C,  in  its  final  configuration,  xBus  serves  more  than  15  client-server 
subsystem  and  brings  all  the  available  information  in  the  central  interface  Sauvim 
Explorer. 

HLC's  objective  is  to  develop  a  supervisory  control  module  that  will  minimize  human 
involvement  in  the  control  of  the  underwater  vehicle  and  its  manipulation  tasks.  In  the 
gradual  passage  from  human  tele-operated  manipulation  to  autonomous  intervention, 
the  most  noticeable  aspect  is  the  increase  of  the  level  of  information  exchanged  between 
the  system  and  the  human  supervisor.  In  teleoperation  with  ROVs,  the  user  sends  and 
receives  low  level  information  in  order  to  directly  set  the  position  of  the  manipulator 
with  the  aid  of  a  visual  feedback. 
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As  the  system  becomes  more  autonomous,  the  user  may  provide  only  a  few  higher  level 
decisional  commands,  such  as  "unplug  the  connector",  interacting  only  with  a  higher 
level  task-description  layer.  The  management  of  lower  level  functions  (i.e.  driving  the 
motors  to  achieve  a  particular  task)  is  left  to  the  onboard  system.  The  level  of  autonomy 
is  related  to  the  level  of  information  needed  by  the  system  in  performing  the  particular 
intervention. 

With  the  above  considerations  in  mind,  the  HLC  module  initially  involved  the 
development  of  high-level  task  planning  where  a  mission  is  always  composed  of  two 
parts:  the  goal  and  the  method  of  accomplishment.  In  other  words,  "what  do  I  need  to 
do"  and  "how  do  I  do  it."  Following  this  strategy,  a  new  high-level  architecture  of 
vehicle  control,  named  the  Intelligent  Task-Oriented  Control  Architecture  (ITOCA),  was 
developed  for  SAUVIM. 

In  phase  III-B  there  was  a  major  upgrade  of  this  configuration.  The  high  level  control 
layer  of  both  the  manipulation  and  the  navigation  systems  have  been  standardized  and 
upgraded  to  a  powerful  custom  programming  language. 

A  software  emulated  CPU,  where  the  mission  control  resides,  hosts  this  new  dedicated 
programming  language  developed  in  order  to  address  the  above  issues  [Marani05].  This 
language,  suitable  for  real-time  embedded  control  systems,  offers  at  the  same  time 
flexibility,  good  performance,  and  simplicity  in  describing  a  generic  complex  task.  Its 
layer  abstraction  approach  allows  an  easy  adaptation  to  the  hardware-specific 
requirements  of  different  platforms.  For  example,  the  same  module  can  be  found  in  the 
manipulator  platform  for  describing  a  generic  manipulation  task  and  in  the  main 
navigation  controller  for  driving  the  vehicle  to  the  target  area.  The  client-server 
approach  allows  the  necessary  communications  between  the  arm  and  the  navigation 
module. 

The  language  is  completely  math-oriented  and  capable  of  symbolic  manipulation  of 
mathematical  expressions.  The  last  is  an  important  distinctiveness  from  most  of  the 
currently  available  robot  programming  languages.  The  procedural  approach  has  been 
chosen  in  order  to  enhance  the  performance  while  maintaining  the  flexibility  required 
for  executing  complex  tasks.  It  is  particularly  suitable  for  real-time  embedded  systems, 
where  the  interaction  of  a  generic  algorithm  with  the  time  is  critical. 

The  Phase  III-C  made  an  extensive  use  of  the  above  environment,  with  several  upgrade 
of  the  language,  and  an  important  addition  of  the  Central  Intelligence  Data  Manifold 
module.  The  latter  is  a  third  environment  that  hosts  a  higher  level  decisional  layer,  for 
coordinating  the  priorities  between  the  vehicle  and  arm  MLC  subsystems. 


Predictive  Virtual  Environment  (PVE)  -  The  PVE  was  aimed  at  developing  a 
supervisory  monitoring  system  for  SAUVIM  to  smoothly  and  realistically  integrate 
mapping  data  with  on-line  sensory  information  even  in  the  midst  of  delayed  and  limited 
information.  This  virtual  reality  (VR)  based  system  must  also  be  able  to  accurately 
predict  the  current  status  and  location  of  the  vehicle  under  these  conditions.  The 
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development  for  the  PVE  has  been  modular.  The  various  modules  are:  the  SAUVIM 
Simulation  Software  (SSS);  the  SAUVIM  Video  Overlay  Software  (SVOS);  the 
Communication  Software  (CS);  and  the  artificial  neural  network  (ANN)  Video 
Prediction  Software  (VPS).  In  the  Phases  I  and  II  of  SAUVIM  the  SSS  has  been  upgraded 
from  its  Version  1  to  Version  1.1,  which  includes  the  incorporation  of  a  Magellan 
spaceball  mouse,  an  accurate  3D  graphical  model  of  SAUVIM  and  the  Maris  7080 
manipulator,  scene-smoothing  methods  using  interpolation  techniques,  and  an  easy-to- 
use  user  interface.  The  SVOS  was  developed  to  overlay  video  images  of  the  seafloor 
(texture  and  color)  to  the  graphic  images  to  provide  a  more  accurate  monitoring  of  the 
vehicle,  manipulator  and  environment.  The  CS  for  SAUVIM  was  an  extension  of  the 
NSF's  DVECS  (Distributed  Virtual  Environment  Collaborative  Simulator)  project.  At 
that  time,  the  DVECS  system  used  a  cellular  phone  to  communicate  the  vehicle  data 
from  the  test-site  to  the  monitoring  computer  located  on  campus  for  data  fusion. 
Experiments  have  been  conducted  with  the  ODIN  AUV.  The  experiments  of  ODIN 
were  projected  via  an  ElectroHome  Marquee  8500  CRT  projector  coupled  with  multiple 
Stereographies  (SG)  emitters  and  SG  CrystalEyes  glasses.  Finally,  the  VPS  has  been 
tested,  and,  although  in  its  early  stage,  with  positive  results. 

Successively,  due  to  the  high  maintenance  costs  of  SGI  workstations,  the  overall  virtual 
reality  and  monitoring  system,  which  includes  the  video  prediction,  has  been 
transformed  to  a  much  more  stable  and  inexpensive  personal  computing  system,  taking 
advantage  of  the  emerging  market  of  high  performance  hardware  video  accelerators 
(mostly  targeted  to  PC  games). 

MarisGL  was,  during  the  Phase  II,  the  preliminary  version  of  the  virtual  environment 
targeted  to  the  MARIS  7080  Manipulator  and  making  use  of  a  standard  OpenGL  PC 
video  accelerator.  During  the  Phase  III-A  the  application  was  extended  in  order  to 
introduce  the  vehicle  model,  mainly  for  collision  avoidance  verification.  But  the  most 
important  transition  toward  the  global  virtual  environment  happened  in  the  Phase  III-B 
and  the  current  III-C. 

Here,  the  name  of  the  application,  once  targeted  to  visualize  only  the  configuration  of 
the  arm,  has  been  changed  to  Sauvim  Explorer  (Figure  C).  Sauvim  Explorer  collects  in  a 
unified  application  the  data  from  all  the  sensors  of  SAUVIM,  including  data  from  the 
DIDSON  that  can  be  overlaid  over  the  graphical  reconstruction  of  the  floor. 

It  also  hosts  the  remote  console  clients  for  both  the  Arm  Programming  Language  and  the 
Sauvim  Programming  Language  servers,  and  may  act  as  remote  control  (ROV  mode) 
when  a  sufficient  bandwidth  channel  is  present.  At  this  aim  Sauvim  Explorer  contains 
software  interface  with  several  input  device  hardware,  including  6  DOF  space 
controllers. 
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Figure  C:  Sauvim  Explorer 


This  represents  an  enormous  step  forward  toward  the  unification  of  the  whole  system, 
since  it  required  a  huge  effort  on  the  standardization  of  the  communication  protocol 
between  every  module  of  SAUVIM  (sensors,  actuators,  controllers...).  With  this  modular 
approach  it  is  now  extremely  easy  to  add  further  sensor  modules  to  SAUVIM  and  add 
their  input  and  outputs  to  the  SE  application  with  a  minimal  effort. 

During  the  last  Phase  III-C  Sauvim  Explorer  was  integrated  with  a  global  data  recorder, 
thus  allowing  data  storing  and  replaying  of  all  the  mission  of  the  vehicle. 


SAUVIM  Design  (SD)  -  This  task  was  one  of  the  main  objectives  of  the  SAUVIM  project. 
It  was  an  effort  to  design  and  develop  efficient,  reliable  hardware/ software  architectures 
of  SAUVIM.  Due  to  the  immense  demand  of  this  task,  it  has  been  divided  into  five  sub¬ 
tasks,  which  are: 

Reliable,  Distributed  Control  (RDC) 

Mission  Sensor  Package  (MSP) 

Hydrodynamic  Drag  Coefficient  Analysis  (HDCA) 

Mechanical  Analysis  and  Fabrication  (MAF) 

Mechanical-Electrical  Design  (MED). 

The  goal  of  RDC  was  to  develop  a  reliable  and  efficient  computing  architecture  for 
signal  and  algorithmic  processing  of  the  entire  SAUVIM  system.  The  proposed  system 
is  a  multi-processor  system  based  on  a  6U  VMEbus  and  the  VxWorks  real-time 
operating  system.  This  system  is  capable  of  high  processing  throughput  and  fault 
tolerance.  Currently  the  system  consists  of: 

Two  VMEbuses,  which  are  the  navigation  control  system  and  the  manipulator 
control  system 
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Two  PC104+  computers  dedicated  to  sensor  data  acquisition,  processing  and 

sharing; 

One  PC104+  that  hosts  the  video  processing  algorithms  for  the  target  detection  and 

tracking  system 

One  PC104+  for  the  ultrasonic  tracker  (currently  in  development). 

The  main  VMEbus,  or  the  navigation  control  system,  has  one  Motorola  MC68060  CPU 
boards  and  a  digital/ analog  I/O  board,  and  two  Pentium-M  processor-based  PC104+ 
boards,  which  share  data  through  the  Ethernet-based  standard  protocol  xBus.  The 
navigation  control  system  handles  the  communication,  supervision,  planning,  low-level 
control,  self-diagnostics,  video  imaging,  etc. 

The  second  VMEbus,  or  the  manipulator  control  system,  has  one  Motorola  MC68060 
CPU  and  several  hardware-dedicated  I/O  board.  One  PC104  board  aids  the 
manipulator  control  system  in  performing  the  video  processing  operation  necessary  to 
detect  and  track  the  target.  Data  resulting  from  the  video  processing  subsystem  are 
shared  with  the  whole  SAUVIM  system  (including  the  Sauvim  Explorer  interface),  again 
using  the  standard  xBus  protocol. 

The  manipulator  control  system,  once  independent  and  dedicated  to  the  manipulator 
control,  can  now  share  its  programming  language  subsystem  with  the  navigation 
controller,  a  very  important  feature  to  perform  underwater  intervention. 

Many  of  the  hardware  components  have  been  tested  and  are  interfaced  with  its 
respective  software  systems.  Various  optimization  changes  have  been  implemented  to 
minimize  communication  and  computation.  The  overall  hardware  and  software 
architectures  have  been  completed  and  integrated.  Tests  for  the  RTOS  architecture  has 
been  integrated  with  the  SAUVIM  vehicle  hardware  and  tested  as  individual 
components.  The  overall  vehicle  control  with  sensor  feedback  has  been  conducted  at 
Snug  harbor.  This  development  will  continue  throughout  the  vehicle's  development 
process. 

The  objective  of  the  MSP  was  to  provide  semi-continuous  records  of  underwater 
environment  such  as  water  depth,  temperature,  conductivity,  computed  salinity, 
dissolved  oxygen,  magnetic  signature  of  the  seafloor,  pH,  and  turbidity,  during  the 
survey  mode.  In  the  intervention  mode,  the  MSP  also  provides  compositional 
parameters  at  a  selected  seafloor  target,  including  pumped  samples  from  submarine 
seeps  or  vents.  The  MSP  is  an  independent  system  with  its  own  PC  104  CPU  and  its 
own  power  supply  residing  in  a  separate  pressure  vessel.  All  of  the  sensors  have  been 
purchased  and  mounted,  and  an  initial  field  test  at  the  Loihi  Seamount  has  been 
conducted.  Other  tests  have  been  conducted  to  optimize  the  scientific  sensor  data- 
gathering  capabilities.  The  communication  from  the  MSP  and  the  vehicle  CPUs  was 
initially  based  on  RS-232C  serial  link. 

The  HDCA  was  used  to  determine  the  hydrodynamic  coefficients  via  a  numerical 
solution  of  full  Navier-Stokes  equations  using  PHOENICS,  a  commercial  computational 
fluid  dynamics  (CFD)  code.  Initial  results  from  the  PHOENICS  software  have  produced 
mixed  results.  The  current  vehicle  fairing  has  produced  a  drag  coefficient  of  0.40; 
however,  it  has  not  yet  been  verified.  Other  CFD  software  and  model  testing  is  being 
conducted  to  verify  the  drag  coefficient  results  before  the  implementation  of  the  vehicle 
fairing  on  SAUVIM.  There  has  been  no  significant  development  in  this  task  group.  The 
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hydrodynamic  coefficients  will  be  obtained  through  vehicle  motion  experiments  in  the 
near  future  to  aid  in  simulator  developments. 


The  MAF  had  three  objectives.  Its  primary  goal  is  to  design  and  fabricate  composite 
pressure  vessels  with  end  caps  and  connector  openings  for  full  ocean  depths  taking 
stress,  buckling,  hydrothermal  effects,  and  fatigue  analysis  into  account;  and  its  two 
secondary  goals  are  to  design  and  fabricate  the  SAUVIM  fairing  and  to  analyze  the 
SAUVIM  frame.  A  thorough  analysis  and  comparison  of  the  Ti-6A1-4V,  AS4/ Epoxy, 
and  AS4/PEEK  pressure  vessels  manifest  the  advantage  of  composite  materials  in 
reduction  of  weight,  size  and  strength.  Using  these  results,  a  scaled  model  prototype 
using  AS4/PEEK  has  been  fabricated  and  tested.  A  1/3  sized  prototype  is  being 
fabricated  and  will  also  be  tested.  For  the  shallow  water  vehicle  test,  a  full-sized, 
fiberglass  pressure  vessel  with  aluminum  end  caps  have  been  manufactured  and  tested. 
These  vessels  are  being  used  to  determine  the  final  hardware  layout.  The  aluminum 
frame  has  been  designed  and  fabricated.  A  full-ocean  depth  pressure  vessel  of 
AS4/PEEK  has  been  developed  and  tested.  However,  due  to  several  unknowns 
regarding  composite  pressure  vessels,  the  vehicle  has  been  equipped  with  1000  meter- 
depth  aluminum  pressure  housing.  These  aluminum  housings  will  be  used  for  the 
shallow  and  mid  water  depth  experiments.  The  fairing  analysis  has  been  developed  and 
expanded.  After  exploring  various  manufacturing  and  molding  methods,  the  initial 
fairing  was  fabricated  in-house  in  Phase  II-B. 

The  MED  was  the  integration  of  the  mechanical  and  electrical  components  for  SAUVIM. 
First,  the  design  specifications  were  established  for  the  fairing,  frame,  instrument 
pressure  vessels,  buoyancy  systems,  mission  sensor,  passive  arm  and  robotic 
manipulator  tasks.  Second,  after  scrutinizing  review  of  SAUVIM' s  major  components  - 
i.e.  sensors,  actuators  and  infrastructure  -  in  terms  of  power  consumption,  compatibility, 
weight  distribution,  buoyancy  distribution,  hydrodynamic  effects  and  task  effectiveness, 
all  major  components  have  been  purchased.  Technical  drawings  of  the  vehicle  frame, 
fairing,  and  related  sub-structures  have  been  completed.  Most  of  the  mechanical  and 
electrical  components  have  been  fabricated  and  integrated  with  the  overall  electrical 
layouts.  There  were  two  wet-tests  in  Phase  II- A,  several  autonomous  shallow  water 
tests  were  conducted  in  Phase  II-B,  and,  from  Phase  II-C  onwards,  several  vehicle 
navigation  and  underwater  manipulation  works. 
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The  main  body  of  this  report  is  devoted  to  the  detailed  descriptions  of  the  major  technical 
developments  and  achievements  during  the  period  of  Phase  III-C,  from  October  of  2006  to 
the  October  2009. 


A  detailed  description  of  the  work  of  the  prior  phases  was  given  in  the  previous  SAUVIM 
final  reports. 


Giacomo  Marani 

SAUVIM  Acting  Principal  Investigator 
January  28,  2010 
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Figure  D:  SAUVIM:  Simplified  Gantt  Chart 
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SAUVIM  Final  Demonstration:  the  first  fully 
autonomous  mission 


Project  Leader (s):  Giacomo  Marani 

Personnel:  Giacomo  Marani,  Song  K.  Choi,  Aaron  Hanai,  Kaikala  Rosa 

Objectives 

The  aim  of  the  SAUVIM  final  experiment  is  to  demonstrate  the  capabilities  of  the  overall 
system,  with  particular  care  to  the  autonomy  aspect. 

Introduction 

The  first  fully  autonomous  underwater  manipulation  in  an  unstructured  environment  was 
demonstrated  at  the  Snug  Harbor,  Honolulu,  Hawaii  at  the  end  of  the  project.  The  SAUVIM 
vehicle  performed  a  fully  autonomous  navigation  and  manipulation  task.  This  live 
demonstration  presents  a  technological  breakthrough  in  the  field,  as  the  autonomous 
manipulation  had  been  a  bottleneck  for  underwater  intervention  missions. 


The  SAUVIM  in  water  first  performed  the  self-calibration  routine,  initializing  its  sub¬ 
systems.  After  the  calibration,  the  SAUVIM  started  its  motion  for  a  pre-given  mission, 
which  was  to  search  for  an  underwater  object  and  to  securely  hook  a  recovery  device  the 
object. 
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The  object's  location  was  roughly  given.  It  was  know  to  lie  somewhere  in  the  front  of  the 
platform  above.  The  platform  location  was  submerged  in  an  unknown  location  of  the  SNUG 
harbor. 
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Once  the  target  was  detected,  the  vehicle  approached  to  it  and  positioned  itself  for  the 
optimized  manipulation.  While  the  vehicle  was  floating  in  the  water  column,  using  the 
unified  coordinated  motion  control  of  the  vehicle  and  manipulator  system,  the  vehicle 
performed  an  autonomous  manipulation  that  was  tagging-the-object.  After  completing  the 
mission,  the  vehicle  came  back  to  the  dock  by  using  feature-based  navigation.  The  whole 
sequence  was  autonomously  done  and  the  same  mission  was  successfully  repeated  four 
times. 

Phase  1:  Undock 

In  this  initial  phase  SAUVIM  leaves  the  pier,  where  is  normally  docked,  to  reach  the  center 
of  the  harbor. 


SNUG  Harbor 


The  navigation,  in  this  undocking  phase,  was  aided  by  the  DGPS,  which  in  our  case  was 
giving  an  accuracy  of  about  2-3  meters.  This  accuracy  is  more  than  sufficient  to  select  a 
target  area  where  to  begin  the  search  for  the  platform. 


Phase  2:  Platform  Search 

Once  reached  the  center  area  of  the  harbor,  SAUVIM  begins  searching  for  the  main 
platform.  In  this  phase  it  uses  the  DIDSON  sonar  as  explained  in  the  DIDSON  section. 
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After  detecting  the  platform,  SAUVIM  aligns  itself  to  it,  preparing  for  the  dive  and  the 
autonomous  approach. 


Phase  3:  Navigate  and  dive  toward  the  platform 

After  the  precise  location  of  the  platform  has  been  measured  in  the  previous  phase,  the 
vehicle  initiates  the  approaching  phase. 


Using  the  PHINS  corrected  by  the  DVL,  SAUVIM  performs  dead  reckoning  in  order  to 
position  itself  just  30  cm  above  the  platform. 
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The  above  snapshot  shows  a  view  from  the  SAUVIM  cameras  while  approaching  the 
platform.  Note  the  top-left  camera:  on  different  trials,  the  platform  bar  has  been  always 
centered  in  the  view  with  a  standard  deviation  of  about  10  cm.  This  is  a  very  impressive 
result  in  underwater  object  identification,  which  I  believe  has  never  been  accomplished  with 
the  DIDSON  sonar. 

Phase  4:  Hovering 

SAUVIM  is  equipped  with  an  advanced  navigation  control  capable  of  precise  station 
keeping  (hovering).  This  is  accomplished  with  a  6  DOF  dynamic  controller  and  an  extended 
Kalman  filter  for  real-time  identification  of  the  center  of  buoyancy  (COB). 

The  COB  identification  is  a  remarkable  and  unique  feature  of  the  system:  it  allows  to 
maintain  active  the  control  on  pitch  and  yaw  while  minimizing  the  energy  required.  As  a 
matter  of  fact,  the  reference  position  for  roll  an  pitch  is  chosen  in  order  to  align  the  COB 
over  the  center  of  mass  (COM). 

The  video  shows  the  impressive  stability  and  a  very  precise  position  control,  in  the  order  of 
sub-centimeter  in  translation.  In  particular,  note  the  stability  in  pitch  and  yaw,  a  clear  sign 
that  the  controller  is  active.  This  is  a  fundamental  requirement  for  stabilizing  the  vehicle 
during  manipulation. 
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Phase  5:  Preparing  the  arm 

SAUVIM  is  now  hovering  above  the  main  platform,  and  the  short-range  search  for  the 
target  is  about  to  begin.  During  the  previous  navigation,  the  arm  was  safely  docked  in  the 
bay  area.  Now  it  is  time  to  wake,  pick-up  the  recovering  tool  and  fully  undock  ready  for  the 
final  search. 
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Phase  6:  Autonomous  Manipulation 


Finally  the  most  distinctive  feature  of  the  AUV:  the  capability  of  performing  autonomous 
manipulation  tasks  in  the  underwater  environment. 

In  this  final  phase  the  manipulator  starts  searching  for  the  target  using  the  camera  that 
carries  (short-range  optical  vision).  The  video  below  shows  also  the  real-time  processed 
view  from  the  arm  camera. 


Once  the  target  has  been  detected,  MARIS  enters  in  a  tracking  mode  and,  after  successfully 
locking  the  target,  performs  few  movements  finalized  to  hook  the  carabineer.  Note  that  the 
carabineer  is  a  modified  one,  without  the  front  part,  so  that  we  could  repeat  the  task  several 
time  without  the  need  to  undo  the  hooking. 

One  important  feature  in  this  phase  is  the  workspace  optimization.  The  vehicle  is  set  to 
adjust  its  position  in  real  time  in  order  to  optimize  the  manipulation.  This  it  is  not  really 
noticeable  from  the  previous  image,  since  the  target  was  luckily  found  in  a  good  position  for 
the  arm.  Instead,  in  following  snapshot,  the  target  was  found  to  the  margin  of  the 
workspace.  This  time,  the  adjustment  in  the  vehicle  position  was  more  evident  and  at  the 
end  the  arm  was  not  in  the  fully  stretched  position. 
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Note  the  excellent  stability  of  the  vehice,  while  performing  the  autonomous  manipulation  in 
hovering. 

Phase  7:  Return  to  the  pier 

Finally,  with  the  task  successfully  accomplished,  the  vehicle  begun  its  way  back  to  the 
docking  pier.  Precise  docking  with  the  GPS  is  of  course  unfeasible,  because  the  accuracy  is 
limited  to  2  meters. 

Here,  the  vehicle  showed  another  remarkable  feature:  feature-based  navigation.  After 
surfacing,  SAUVIM  went  to  the  center  of  the  harbor  looking  for  again  for  the  submerged 
platform.  Then,  knowing  the  relative  distance  of  the  docking  pier  with  respect  to  this 
reference,  SAUVIM  set  the  course  for  its  home.  With  this  method,  the  standard  deviation  of 
the  final  position  is  about  10  cm,  a  respectable  result  in  this  kind  of  tasks. 
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Adaptive,  Intelligent  Motion  Planning  (AIMP) 


Project  Leader (s): 
Personnel: 

Past  Project  Leader (s): 
Past  Personnel: 


Giacomo  Marani 
Giacomo  Marani 

Dr.  Tae  Won  Kim,  Dr.  Kazuo  Sugihara  &  Dr.  Song  K.  Choi 
Mr.  John  Smith,  Dr.  Shenyan  Zhen,  Mr.  Haidong  Chang,  Ms. 
Hongshi  Chen,  Mr.  Xihua  Xu,  Mr.  Dwayne  Richardson,  Mr.  Sonny 
Kim.  Mr.  Jangwon  Lee  &  Mr.  Yongcan  Zhang 


Objectives 


This  sub-project  aims  at  developing  the  motion  planning  system  for  SAUVIM.  It  is 
intelligent  and  adaptive  in  the  sense  that  the  system  is  capable  of  decision-making  at  a  task 
or  mission  level  and  can  deal  with  an  unknown  or  time-varying  environment. 

There  are  three  basic  objectives. 

•  To  develop  an  off-line  3D  motion  planning  algorithm. 

•  To  develop  an  on-line  3D  motion  planning  algorithm. 

•  To  develop  an  adaptive,  intelligent  motion  planning  system  by  integrating  the  off¬ 
line  and  the  on-line  planning  algorithms. 


Current  Status  (Tasks  Completed) 


The  task  has  been  completed  in  the  previous  phases.  Refer  to  previous  reports  for  its 
descriptions. 
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Automatic  Object  Ranging  and  Dimensioning 
(AORD) 

Project  Leader (s):  Dr.  Giacomo  Marani 

Personnel:  Mr.  Luca  Gambella,  Dr.  Giacomo  Marani 

Past  Project  Leader(s):  Dr.  Son-Cheol  Yu,  Dr.  Tae  Won  Kim,  Dr.  Junku  Yuh,  &  Dr.  Curtis 

S.  Ikehara 

Past  Personnel:  Dr.  Tae  Won  Kim,  Mr.  Marc  Rosen,  Mr.  Mike  Kobayakawa,  Mr. 

Henrik  Andreasson  &  Mr.  Anders  Andreasson,  Mr.  Aaron  Hanai, 
&  Mr.  Oliver  Easterday 


Objectives 


The  main  objective  of  the  AORD  is  to  develop  a  multiple  sensor  system  to  be  utilized  during 
SALT VIM's  intervention  missions  to  locate  the  target.  The  system  will  allow  accurate  vehicle 
positioning,  workspace  dimensioning  and  ranging,  and  manipulator  homing  to  the  task 
object.  The  localization  task,  that  is  the  main  support  for  the  capabilities  of  the  autonomous 
manipulation  of  SAUVIM,  is  performed  by  using  and  fusing  different  technologies 
(acoustical  and  optical)  in  order  to  guarantee  a  suitable,  range  dependent,  level  of  reliability, 
precision  and  accuracy 


Current  Status  (Tasks  Completed  during  10/25/2006  -  10/30/2009) 


Overview 

The  original  idea  of  the  homing  sensor  was  to  use  a  dedicated  PC104  computer  with  camera 
to  detect  a  simple  circular  barcode.  It  was  originally  tested  to  confirm  its  performance  in  the 
water,  and,  despite  results  were  good  enough  to  use  the  bar  code  in  the  water,  it  suffered  of 
obvious  application  limitations.  During  the  past  years,  and  especially  starting  with  the 
Phase  III-B,  the  idea  shifted  toward  a  more  organized  and  range  dependant  Target 
Identification  procedure. 

The  localization  subsystem,  that  is  the  main  support  for  the  capabilities  of  the  autonomous 
manipulation  of  SAUVIM,  is  performed  by  using  and  fusing  different  technologies 
(acoustical  and  optical)  in  order  to  guarantee  a  suitable,  range  dependent,  level  of  reliability, 
precision  and  accuracy.  The  SAUVIM  AUV  switches  through  three  main  sensing  methods  in 
order  to  acquire  reliable  data.  As  shown  in  Figure  B,  the  sensor  technology  changes 
according  to  the  combination  of  range  and  accuracy  needed. 

In  long  range  (over  25m),  375KHz  image  sonars  are  used  for  initial  object  searching.  The 
accuracy  in  this  range  is  necessary  only  to  direct  the  vehicle  toward  the  target  zone. 

In  mid-range  (2-25m),  a  Dual  frequency  IDentification  SONar  (DIDSON)  is  used  for  object 
recognition  and  the  vehicle  positioning.  This  is  the  phase  where  the  vehicle  has  to  position 
itself  in  order  to  have  the  target  confined  within  the  manipulation  workspace.  At  this  range. 


32 


and  in  case  of  turbid  water,  it  is  virtually  impossible  to  use  conventional  optical  cameras  to 
identify  an  object.  This  justify  the  use  of  the  DIDSON,  which  has  been  used  as  a  ranging 
sensor  from  Phase  II-C  onwards.  During  the  Phase  III-B  our  focus  has  been  directed  toward 
refining  the  algorithms  of  autonomous  target  identification  with  the  DIDSON,  thus  allowing 
the  SAUVIM  vehicle  to  find  a  path  toward  the  target  area. 


Target 

Short  range: 

Camera,  Motion 
Tracker 

Medium  range: 

DIDSON 

Long  range: 

Sidescan  sonar, 
Imagingsonar 


Figure  AORD-1.  The  phases  involved  in  a  search  for  the  target. 

Finally,  when  the  target  is  within  the  manipulator  workspace,  short  range  and  high  accuracy 
sensor  are  used  in  order  to  perform  the  actual  intervention  task.  This  goal  is  achieved  with 
the  combined  use  of  underwater  video  cameras  and  the  ultrasonic  motion  tracker  described 
above,  used  to  retrieve  the  real-time  6  DOF  position  of  the  target  during  the  manipulation 
tasks.  The  device  utilizes  high  frequency  sound  waves  to  track  a  target  array  of  ultrasonic 
receivers.  The  use  of  4  transmitters  at  the  stationary  positions  with  4  receivers  on  the  target 
can  be  used  to  determine  the  6  DOF  generalized  position  (rotation  and  translation)  of  the 
object. 

During  the  Phase  III-C  the  main  effort  was  to  upgrade  and  implement  the  medium  range 
object  identification  with  the  DIDSON,  for  guiding  the  vehicle  toward  the  target  area.  Based 
on  the  use  of  the  DISDON  sonar,  the  goal  is  to  acquire  the  Earth-referenced  cartesian 
coordinates  of  a  known  target,  with  the  necessary  accuracy  required  for  positioning  the 
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vehicle  so  that  the  target  falls  within  the  manipulator  workspace..  The  task  has  been 
successfully  accomplished,  leading  to  the  realization  of  a  robust  and  repeatable  module  for 
guiding  the  vehicle  toward  the  target. 

In  order  to  achieve  the  above  objective,  extensive  product  engineering  works  have  been 
necessary,  other  than  several  further  developments  of  the  hardware/ software  control 
system. 

The  final  objectives  include  the  following: 

•  Development  of  the  Ethernet  communication  protocol  for  the  DIDSON 
parameter  setting,  sensor  control  and  management  and  data  acquisition  with 
extensive  lab  testing  in  order  to  verify  the  consistence  and  stability  of  the  data 
exchange  in  a  structured  known  environment. 

•  Development  of  theoretical  solutions  for  the  DIDSON  data  processing,  user- 
friendly  visualization,  model  estimation  and  target  identification  tasks. 

•  Development  of  software  for  the  DIDSON  management  which  includes  a  low 
level  sensor  management  and  model  estimation  and  high  level  user 
visualization,  bottom  exploration  and  target  identification.  This  requires 
extensive  testing  in  a  real  sea  water  environment. 

•  Integration  of  the  DIDSON  sonar  on  the  vehicle. 

The  final  step,  after  the  above  developments,  was  the  first  underwater  middle  range 
exploration,  target  identification  and  localization  experimented  by  DIDSON  sonar. 

Summary 

1.  DIDSON  sonar  main  features: 

•  Specifications 

•  Acquisition  characteristics 

2.  DIDSON  sonar  data  exchange  protocol: 

•  DIDSON  communication  and  management 

3.  DIDSON  sonar  SAUVIM  applications 

4.  Fine  middle  range  exploration: 

•  Development  of  drivers  and  interface  software  for  the  DIDSON  sonar  SAUVIM  xBus 
framework 

•  Development  of  software  for  data  processing  and  user-friendly  visualization 

5.  Object  recognition  and  vehicle  positioning: 

•  Development  of  software  for  model  estimation 

•  Development  of  software  for  target  identification  and  absolute  position  localization 

6.  Underwater  DIDSON  tests: 

•  Bottom  mapping 

•  Known  object  identification  and  localization 
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Didson  sonar  features 


The  DIDSON  (Dual  frequency  IDentification  SONar,  Figure  IDUTI-2)  is  a  sonar  with 
acoustic  lenses  that  operates  at  two  frequency,  1  MHz  and  1.8  MHz,  at  an  operative  range 
up  to  40m.  Its  dimensions  are  30.7  cm  long  by  20.6  cm  high  by  17.1  cm  wide  with  a  weight  in 
air  7  kg  while  in  water  0.6  kg;  the  housing  is  suitable  for  operating  up  to  more  than  152  m 
deep  (see  [1],  [5]). 

The  device  uses  acoustic  lenses  to  form  very  narrow  beams  during  transmission  of  pulses 
and  reception  of  their  echoes.  This  hardware  solution  has  the  advantage  of  low  power 
consumption:  in  fact,  no  power  for  beamforming  is  required  and  it  uses  only  30  watts  to 
operate;  this  is  an  important  feature  for  the  power  budget  of  an  AUV.  A  second  advantage  is 
the  ease  to  transmit  and  receive  from  the  same  beam.  Images  are  also  cleaner  and  sharper 
thanks  to  the  reduced  acoustic  cross  talk  and  the  higher  resolution. 

The  DIDSON  is  physically  composed  by  a  linear  array  of  transducers  that  covers  a  cone  of 
12  degrees  height  and  29  degrees  width  respectively,  using  96  elements  regularly  spaced 
each  0.3  degrees:  in  high  frequency  all  the  set  of  transducers  is  used  while  in  low  frequency 
only  even  lenses  are  used  (half  set  of  transducers).  In  both  cases,  the  frame  rate  is  up  to  20 
frame/  s. 

The  sonar  is  connected  and  control  through  Ethernet  connection  using  its  Windows 
application;  moreover,  a  NTCS  video  output  is  available  only  for  data  visualization. 

The  aim  of  this  work  was  to  use  the  DIDSON  within  SAUVIM  framework  in  a  fast,  direct 
and  reliable  way  with  a  full  control  of  the  device  and  of  the  collected  data:  in  order  to 
achieve  such  result  an  alternative  way  to  the  Windows  application  has  been  realized. 


Figure  IDUTI-2.  The  DIDSON  (Dual  frequency  IDentification  SONar) 
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Didson  control  protocol 

The  DIDSON  is  controlled  via  a  client-server  model  through  Ethernet  connection  and  UDP 
protocol:  the  user  (client)  asks  data  or  sets  parameters  to  the  server  (DIDSON)  that  replays 
consistently. 

In  the  client  to  server  direction  communication,  parameters  are  set  and  data  are  asked. 
Relevant  parameters  are: 

Frequency:  1  MHz  or  1.8  MHz 

Window  Start:  distance  of  the  closest  acoustic  return 

Window  Length:  range  of  the  acoustic  set  of  return  for  a  single  beam 
Once  asking  for  data  the  server  replays  to  the  client  (server  to  client  direction 
communication)  with  a  set  of  data.  The  quantity  of  data  depends  on  the  frequency:  a  header 
of  256  bytes  is  followed  by  24576  bytes  of  acoustic  returns  in  low  frequency  or  49152  bytes 
in  high  frequency.  The  header  contains  a  set  of  DIDSON  information  and  checking  flags 
that  are  listed  hereafter: 

unsigned  short  cmd; 
unsigned  short  size; 
unsigned  short  pkt_code; 
unsigned  short  pkt_num; 
unsigned  int  framenumber; 
unsigned  int  frametime  [2]; 
unsigned  int  version; 
unsigned  int  status; 
unsigned  int  year; 
unsigned  int  month; 
unsigned  int  day; 
unsigned  int  hour; 
unsigned  int  minute; 
unsigned  int  second; 
unsigned  int  hsecond; 
unsigned  int  hi_res; 
unsigned  int  window_start; 
unsigned  int  window_length; 
unsigned  int  threshold; 
unsigned  int  intensity; 
unsigned  int  gain; 
unsigned  int  degCl; 
unsigned  int  degC2; 
unsigned  int  humid; 
unsigned  int  focus; 
unsigned  int  battery; 
char  stringl[16]; 
char  string2[16]; 
float  velocity; 
float  depth; 
float  altitude; 
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float  pitch; 

float  pitchrate; 

float  roll; 

float  rollrate; 

float  heading; 

float  headingrate; 

float  sonarpan; 

float  sonartilt; 

float  sonarroll; 

double  latitude; 

double  longitude; 

float  sonarposition; 

unsigned  int  config_flags; 

float  beamtilt; 

float  targetrange; 

float  targetbearing; 

unsigned  int  targetpresent; 

unsigned  int  firmwarerevision; 

unsigned  int  m_nFlags; 

unsigned  char  rsvd[24]; 

unsigned  char  user[12]; 

Following  the  header,  the  acoustic  data  itself  is  stored  in  a  byte  array  of  dimensions  equals 
to  sample  x  beam,  where  sample  ranges  from  0  to  511  and  beam  ranges  from  0  to  47  (in  low 
frequency)  or  from  0  to  95  (in  high  frequency).  The  client-server  module  has  been 
implemented  in  C++  in  order  to  realize  suitable  interface  driver  for  the  DIDSON  sensor 
integration  within  the  SAUVIM  xBus  framework. 

Such  data  are  the  raw  polar  information  used  to  generate  the  DIDSON  acoustic  image.  Each 
value  corresponds  to  the  amplitude  of  the  acoustic  beam  return  at  a  certain  distance:  so  it 
refers  to  a  volume  of  water  as  bigger  as  farer  from  the  transducer  it  is  (each  beam  is  12 
degrees  high  and  0.3  degrees  width).  Moreover  due  to  the  beam  high,  two  objects  at  the 
same  distance  from  the  sonar  and  one  above  the  other  are  indiscernible  because  they  belong 
to  the  same  volume  of  water  so  they  correspond  to  the  same  return  value:  the  information 
about  the  12  degree  beam  high  is  compressed  in  one  return  causing  lack  of  information  and 
uncertainty  in  the  polar  to  Cartesian  data  processing.  Another  consideration  that  makes 
acoustic  images  different  from  optical  images  depends  on  the  use  of  sound  instead  of  light: 
the  sonar  must  be  oriented  to  project  beams  with  a  small  angle  of  incidence  to  the  object  of 
interest  rather  than  perpendicular  as  usual  optic  vision  suggests.  Such  configuration 
guarantees  the  best  and  more  informative  acoustic  returns  and  as  result  the  object  looks  like 
it  is  seen  from  a  perpendicular  direction  to  the  surface  of  the  object  with  a  shadow  on  the 
back.  The  same  view  with  an  optical  camera  is  achieved  by  orienting  the  camera 
perpendicular  to  the  surface:  such  orientation  using  an  acoustic  camera,  instead,  produces 
an  image  with  a  single  line  perpendicular  to  the  center  beam  axis  because  each  beam  hits  the 
surface  at  the  range  of  the  perpendicular  surface  in  only  a  point. 
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These  considerations  bring  to  a  deformation  of  object  shape  according  to  the  angle  of 
incidence  of  beams  array;  the  dimension  of  the  shape  is,  instead,  free  from  the  range  on  the 
contrary  of  optical  vision  where  object  are  as  smaller  as  farer  the  source  of  view  is. 

So,  during  the  data  processing  phase  these  factors  have  to  be  considered  in  order  to 
implement  a  suitable  data  visualization  technique: 

Polar  representation  of  the  raw  collected  data 
12°  vertical  high  compressed  to  2dimensional  XY  plan 
Beams  angle  of  incidence 
Environmental  acoustic  noise 

Figure  IDUTI-3  shows  an  example  of  image  acquisition  by  DIDSON  (left)  and  the 
correspondent  deformation  of  the  insonified  object  (right). 
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Didson  applications 

The  DIDSON  is  a  very  powerful  device  for  underwater  exploration  in  poor  optical  visibility 
conditions  thanks  to  the  acoustic  transducers  use  [3].  Fine  exploration  and  target 
identification  are  important  requirement  for  the  autonomous  navigation  and  control  of 
SAUVIM,  therefore,  by  using  such  device  two  main  applications  have  been  realized: 

•  Fine  middle-range  exploration  of  the  sea  bottom 

•  Object  identification  and  vehicle  positioning 

Fine  middle-range  exploration 

Once  the  data  from  the  DIDSON  (we  refer  to  this  data  as  "polar  pixel")  is  available  via  the 
ad-hoc  interface  driver,  according  to  the  consideration  made  in  the  previous  section,  the 
acoustic  returns  have  to  be  processed  in  order  to  achieve  a  coherent  representation  of  the  sea 
bottom.  The  first  step  at  this  phase  is  the  polar  to  Cartesian  bi-dimensional  conversion  of  the 
coordinates  system  following  by  image  processing  operation  to  rectify  possible 
environmental  noise. 

The  polar  to  Cartesian  conversion  is  performed  by  a  first  dynamical  estimation  of  the 
Cartesian  image  dimensions  in  pixels  and  the  pixel  dimension  as  well  according  to  the 
DIDSON  parameters:  window  start,  window  length,  frequency  (so  number  of  beams)  and 
samples  (set  to  512).  Then  each  Cartesian  pixel  is  associate  to  a  polar  pixel  in  order  to  deal 
with  the  dimension  increase  of  the  volume  associated  to  a  polar  pixel  when  it  moves  away 
from  the  source:  so,  multiple  Cartesian  pixels  correspond  to  the  same  polar  pixel.  Once  the 
Cartesian  image  is  made,  standard  image  processing  techniques  are  used  in  order  to  make 
the  image  clearer,  to  reduce  the  noise  and  to  increase  the  contrast  between  actual  objects  and 
image  background.  Standard  OpenCV  library  of  programming  functions  are  used  at  this 
phase.  Figure  IDUTI-4  shows  an  example  of  raw  polar  image  (left),  of  raw  Cartesian  image 
(center)  and  of  image  processing  result  (right)  on  a  standard  set  of  DIDSON  polar  pixels  (the 
image  is  as  whiter  as  higher  is  the  acoustic  return). 


Figure  IDUTI-4.  Raw  DIDSON  polar  image  (left) 

Raw  DIDSON  Cartesian  image  (center); 
processed  DIDSON  Cartesian  image  (right) 

The  described  procedure  has  also  been  integrated  within  SAUVIM  main  application  (Figure 
IDUTI-5):  this  guarantees  the  DIDSON  parameter  control  and  data  visualization  in  a  user- 
friendly  way  together  with  the  other  SAUVIM  sub-systems. 


Figure  IDUTI-5.  DIDSON  management  panel  of  SAUVIM  main  application 
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Target  identification  and  vehicle  positioning 

Together  with  the  DIDSON  data  visualization,  another  important  task  is  performed  during 
SAUVIM  vehicle  navigation  using  DIDSON  data:  the  identification  and  localization  of 
known  submerged  objects  ([2],  [4])  for  guiding  the  vehicle  to  approach  such  target.  It 
consists  in  recognizing  known  submerged  objects,  in  computing  their  absolute  position  and 
in  using  this  information  in  the  SAUVIM  navigation  control  loop  so  the  vehicle  can 
autonomously  moves  to  the  detected  target. 

In  order  to  use  the  DIDSON  has  a  regular  camera  for  bottom  exploration  and  known  objects 
identification,  two  main  problems  have  to  be  considered: 

Object  shape  deformation  due  to  the  DIDSON  beam  features 

Environmental  noise  that  affects  acoustic  signal  propagation,  so  DIDSON  acoustic 
return  acquisition 

To  deal  with  the  first  problem,  a  recursive  model  estimator  has  been  implemented:  at  a  fixed 
frequency,  a  Cartesian  model  of  the  target  is  built  according  to  object  dimension  (a-priori 
information)  and  DIDSON  orientation  (thanks  to  the  inertial  navigation  system  and  the  pan 
and  tilt  modules  on  board  SAUVIM  vehicle).  So  a  fictitious  DIDSON  image  is  created:  it  is 
an  estimation  of  the  object  Cartesian  image  by  DIDSON  view  point.  Maximum  (white  color) 
acoustic  reflection  from  the  object  surface,  no  reflection  (black  color)  from  the  object  edge 
and  average  reflection  (gray)  from  the  background  are  the  object  model  hypothesis. 

Then,  for  the  target  identification,  standard  image  processing  edge  detection  techniques 
cannot  be  used  due  to  the  shape  deformation  and  environmental  acoustic  noise:  so 
statistical,  bi-dimensional  convolution  based,  approach  via  Intel  Math  Kernel  Library 
(MKL),  that  guarantees  high  performance  for  real  time  application,  has  been  implemented. 
Figure  IDUTI-6  displays  the  whole  target  identification  process  where  the  goal  is  to 
compute  the  target  absolute  position  so  its  absolute  transformation  matrix  which  is  the 
information  to  use  then  in  the  navigation  system  loop: 


Figure  IDUTI-6.  Target  localization  approach 
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Figure  IDUTI-6  shows  two  parallel  processes  running:  DIDSON  data  visualization  and 
target  identification;  in  this  section,  the  second  procedure  is  described  in  detail.  To  have 
these  two  tasks  separated  is  an  important  characteristic  of  the  system.  In  fact,  several 
missions  require  only  exploring  and  mapping  the  sea  bottom  without  any  object  detection 
that  has  different  computational  time:  the  exploration  has  a  15  frame/ sec  refresh  frequency 
while  the  position  of  the  target  is  updated  each  second  due  to  the  image  processing  and 
convolution  operations. 

The  target  localization  is  an  iterative  process  that  recursive  refines  the  target  position 
estimation  using  DIDSON  acoustic  returns;  it  can  be  divided  into  five  steps: 

0.  Initialization:  this  phase  is  important  to  initialize  variables  and  a-priori  information 
of  the  system;  in  particular  the  initial  hypothesis  of  object  positioning  in  DIDSON 
image.  Such  information  is  necessary  to  estimate  the  Target  Relative  transformation 
matrix. 

1.  Target  Absolute  Position  Update:  according  to  the  DIDSON  Absolute  transformation 
matrix  and  the  Target  Relative  (to  the  DIDSON)  transformation  matrix,  the  Target 
Absolute  transformation  matrix  is  computed.  This  matrix  identifies  the  global 
position  of  the  target  (the  goal  of  the  process).  Figure  IDUTI-7  graphically  shows  the 
meaning  of  absolute  transformation  matrix  (O  indicates  the  absolute  coordinate 
system,  P  the  target  coordinate  system) 


Figure  IDUTI-7.  Target  Absolute  Position 

2.  Target  Model  Build:  the  operations  necessary  to  build  a  model  of  the  target  from  the 
DIDSON  view  point  are: 

o  To  build  a  tri-dimensional  representation  of  the  object  surface  in  a  Cartesian 
system  by  points  ( x ,  y,  z)  (see  Figure  IDUTI-9) 
o  To  convert  the  Cartesian  points  into  spherical  points  (r,  6 \  cp) 
o  To  simulate  the  DIDSON  volume  compression  by  reconverting  the  polar 
points  (missing  the  (p  coordinate)  into  bi-dimensional  Cartesian  points  ( x ,  y) 
and  then  gray  scaling  the  obtained  clouds  of  bi-dimensional  points  as  shown 
in  Figure  IDUTI-12 
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Figure  IDUTI-12.  Target  Model  example 


Figure  IDUTI-9.  Coordinate  system 


3.  Bi-dimensional  convolution:  at  this  phase  the  current  DIDSON  Cartesian  image 
(Figure  IDUTI-4)  is  used  and  associated  to  the  target  model  generated  in  the 
previous  step  by  means  of  bi-dimensional  convolution  function  (IDUTI-1).  Bi- 
dimensional  convolution  is  a  very  time  consuming  operation  and  often,  according  to 
the  dimension  of  the  image  to  process,  is  not  applied  to  real  time  and  on  line  tasks. 
Instead,  the  present  implementation  makes  use  of  the  Intel  Math  Kernel  Library 
(MKL)  that  assures  reliable  result  within  300  milliseconds  (using  images  on  the  order 
of  300x600  pixels  and  100x50  pixels).  Figure  IDUTI-14  graphically  shows  the  result  of 
bi-dimensional  convolution  operation  between  two  images  from  real  data,  where  the 
z  coordinate  represents  the  convolution  value  relative  to  a  (v,  y)  position. 

{Ma-\)(Na  -l) 

C(i,j)  =  J  ^A(m,n}  B(i  -  m,j  -  n) 

m= 0  n= 0 

0  <  i  <  Ma  +  Mb  - 1  (IDUTI-1) 

0<j<Na+Nb-l 
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Figure  IDUTI-14.  Convolution  result  visualization 


4.  The  result  of  the  statistical  convolution  between  two  image  indicates  where  the 
images  have  the  better  correlation,  so  where  the  two  image  are  more  similar.  In  the 
present  application,  the  similarity  means  the  presence  of  the  object  in  the  current 
DIDSON  image:  so  the  maximum  of  the  convolution  indicates  where  the  probability 
of  occupation  of  the  object  we  are  looking  for  is  higher.  However,  since  a  maximum 
in  the  convolution  result  always  exists,  in  order  to  determine  if  the  maximum 
corresponds  or  not  to  the  target,  to  use  a  criterion  is  necessary:  the  adopted  criterion 
consists  in  estimating  the  maximum  convolution  result  (it  correspond  to  a  perfect 
correspondence  between  actual  image  and  model)  and  then  set  a  percentage 
threshold  above  which  the  actual  convolution  maximum  is  assumed  to  correspond 
to  the  target. 

The  result  of  the  criterion  determines  the  next  step  on  the  iterative  process:  if  the 
target  is  found,  the  object  absolute  position  is  updated,  so  step  1  is  performed. 
Otherwise,  the  initialization  step  takes  place. 

The  described  procedure  is  repeated  recurrently  during  the  target  identification  task  in 
order  to  have  the  absolute  target  position  constantly  updated  and  guide  the  vehicle  toward 
that  direction  using  the  navigation  system. 

The  implementation  of  the  data  visualization  and  target  identification  algorithms  has  been 
realized  in  C++  programming  language;  moreover,  MatLab  has  been  used  during  test 
phase. 
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Didson  experimental  tests 

In  order  to  test  the  described  procedures  and  algorithms,  several  experiments  have  been 
realized. 

A  first  session  of  DIDSON  tests  in  a  structured  environment  was  necessary  in  order  to  check 
and  control  the  communication  protocol,  the  sensor  management  and  to  collect  reliable 
data.  Then  SAUVIM  vehicle  on  board  tests  followed:  this  step  required  the  hardware 
installation  and  connection  of  the  DIDSON  device  on  the  vehicle  as  showed  in  Figure 
IDUTI-15. 

The  DIDSON  control  and  data  visualization  panel  of  SAUVIM  main  application  is  displayed 
in  Figure  IDUTI-16  where  the  management  of  DIDSON  parameters,  the  visualization  of  the 
acoustic  returns  and  the  result  of  the  target  localization  process  are  accessible. 


Figure  IDUTI-15.  DIDSON  installation  on  SAUVIM 


Data 

visualization  and 
Parameter  setting 


Target  detection 
and 

Localization 


Figure  IDUTI-16.  SAUVIM  application  DIDSON  modules 


45 


Validation 


In  order  to  practically  quantify  the  standard  deviation  of  the  position  error  we  repeated 
several  times  a  pre-defined  experiment,  finalized  to  position  the  vehicle  every  time  in  the 
same  hovering  configuration  above  the  platform.  In  this  experiment,  the  relative  cartesian 
coordinates  of  the  platform  with  respect  to  the  vehicle  were  computed  using  DIDSON 
imagery  taken  at  a  distance  of  10  m  from  the  target.  The  bottom  facing  video  camera  of 
SAUVIM  was  sending  back  images  of  a  known  dimension  feature,  visible  in  the  figure 
below,  where  the  reference  red  line  is  75  mm  in  length. 
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The  same  figure  shows  two  different  final  configurations,  with  only  a  small  difference  in  the 
final  hovering  positions.  Considering  that  the  navigation  from  the  area  where  the  DIDSON 
images  were  taken  to  the  hovering  configuration  was  done  in  dead  reckoning  (thus 
introducing  some  extra  errors),  this  experiment  confirms  an  excellent  repeatability  and,  with 
the  successive  trials,  the  global  standard  deviation  of  the  position  error  has  been  confirmed 
of  the  order  of  few  centimeters. 

The  information  associated  to  the  platform  location  is  made  available  to  the  whole  system, 
included  the  navigation  controller  and  the  main  graphical  interface  (Sauvim  Explorer) 
located  in  the  ground  mission  control  environment.  Figure  IDUTI-17  depicts  a  snapshot  of 
the  mission  interface  during  a  search  test  of  the  submerged  platform.  The  red  rectangle  is 
the  result,  in  real  time,  of  the  iteration  process.  In  the  SAUVIM  missions,  the  platform 
represents  a  submerged  docking  area  of  the  vehicle,  and  the  precise  knowledge  of  its 
location  is  fundamental  to  the  autonomous  landing  procedure. 

The  Sauvim  Explorer  interface  shows  also  the  map  of  the  DIDSON  image  overlapped  to  the 
terrain  profile.  In  case  of  general  imagery,  since  the  The  DIDSON  does  not  disambiguate  the 
elevation  of  the  target,  this  operation  may  result  approximate  In  our  case,  since  we  acquire 
elevation  maps  of  the  seafloor  with  other  sensors,  the  sonar  image  can  unambiguously 
mapped  to  the  cartesian  space. 
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Intelligent,  Coordinated-Motion/ Force  Control 
(ICM/FC) 

Project  Leader (s):  Dr.  Giacomo  Marani 

Past  Project  Leader (s):  Dr.  Junku  Yuh,  Dr.  Tae  Won  Kim,  Dr.  Song  K.  Choi,  Dr.  Kazuo 

Sugihara,  Dr.  Hyun  Taek  Choi,  Mr.  Michael  West  &  Dr.  Nilanjan 
Sarkar 

The  main  technical  development  of  the  ICM/FC  group  is  described  in  the  following 
sections:  Manipulator  Control  and  Test  Platform,  Low-Level  Control,  Active  Feedback 
Thruster  System  (AFTS),  Localization  and  Navigation,  and  High-Level  Control.  The 
Manipulator  Control  and  Test  Platform  is  the  combined  sections  of  the  previous  Theoretical 
Modeling  and  Dry  Test  Design  Set-Up.  The  Localization  and  Navigation  is  a  separation 
from  the  Low-Level  Control  due  to  the  vastness  and  complexity  of  the  research  area. 
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Manipulator  Control  and  Test  Platform  (MCTP) 


Project  Leader (s): 
Personnel: 

Past  Project  Leader (s): 
Past  Personnel: 


Dr.  Giacomo  Marani 

Ms.  Allison  Lyon,  Mr.  Kaikala  Rosa 

Dr.  Song  K.  Choi,  Dr.  Tae  Won  Kim,  Dr.  Junku  Yuh  &  Dr. 
Nilanjan  Sarkar 

Mr.  Tommaso  Bozzo,  Mr.  Gang  Cheng,  Ms.  Jing  Nie,  Mr.  Mike 
Kobayakawa,  Mr.  Mark  Fujita,  Dr.  Gyoung  H.  Kim,  Mr.  Tarun 
Podder,  Mr.  Jin  Hyun  Kim,  Mr.  Jong  Ho  Eun,  Ms.  Stacy  L.  Dees  & 
Mr.  Jangwon  Lee 


Objectives 


During  the  Phase  II  of  SAUVIM,  one  of  the  most  important  objectives  for  the  manipulation 
platform  was  the  first  ocean  test  of  the  system.  In  order  to  achieve  the  above  objective, 
extensive  product  engineering  works  have  been  necessary,  other  than  several  further 
developments  of  the  hardware/ software  control  system. 

The  final  objectives  included  the  following: 

•  Development  of  theoretical  solutions  for  the  arm  control  algorithm  with  extensive  lab 
testing  in  order  to  verify  the  task-space  controller  performances. 

•  Development  of  a  programming  environment  for  manipulators,  which  include  a  low 
level  software-emulated  execution  CPU,  a  high-level  programming  language  and  a 
program  compiler. 

•  Development  and  testing  of  an  ultrasonic-based  tracking  system  for  target  localization. 

•  Development  of  an  extended  subset  of  routines  for  the  arm  programming  environment, 
which  include  a  set  of  calibration  procedures  for  the  joint  offsets  and  the  auto-calibration 
of  the  external  position  sensors. 

•  Development  of  the  arm  parking  procedures 

•  Collision  avoidance  system. 

•  Integration  of  the  manipulator  on  the  vehicle 

•  Development  and  testing  of  a  visual-based  tracking  system  for  close  range  target 
localization 

•  Development  of  an  extended  subset  of  routines  for  the  arm  programming  environment 
which  include  a  set  of  calibration  procedures  for  the  auto-calibration  of  the  camera 

The  final  step,  after  the  above  developments,  was  the  first  underwater  manipulation 
experiment,  as  described  previously. 

These  objectives  have  been  successfully  achieved,  with  good  performances  and  stability.  In 
particular,  the  theoretical  solutions  developed  for  prevent  singularities  showed  an  excellent 
performance  and  were  published  in  several  journal  and  conferences.  Details  on  the  overall 
development  have  been  described  on  the  previous  report  (Phase  II-C,  III- A  and  III-B). 


Current  Status  (Tasks  Completed) 


The  task  has  been  completed  in  the  previous  phases.  Refer  to  previous  reports  for  its 
descriptions. 
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Low-Level  Control  (LLC) 

Project  Leader (s):  Dr.  Giacomo  Marani 

Personnel:  Dr.  Giacomo  Marani 

Past  Project  Leader(s):  Dr.  Side  Zhao,  Dr.  Junku  Yuh,  Dr.  Song  K.  Choi,  Dr.  Tae  Won  Kim 

&  Dr.  Hyun  Taek  Choi 

Past  Personnel:  Ms.  Jing  Nie,  Mr.  Eric  Kardash  &  Mr.  Michael  West 


Objectives 


•  To  design  an  advanced  vehicle  control  for  navigation  and  hovering,  and  coordinated 
motion/ force  control  of  the  vehicle  and  manipulator  during  the  intervention  mode. 

•  To  develop  hybrid  controllers  that  is  robust  to  system  uncertainties  as  well  as  external 
disturbances  of  the  AUV  dynamics. 


Current  Status  (Tasks  Completed  during  10/25/2006  -  10/30/2009) 


In  all  the  SAUVIM  Phases,  most  of  the  vehicle  development  has  been  focused  in  obtaining 
high  performance  in  navigation  and  hovering.  Before  the  past  phase  III-B  the  navigation 
and  hovering  techniques  made  use  of  the  data  from  the  on-board  scan  sonar,  altimeter 
sonar,  inertial  navigation  unit,  Doppler  Velocity  Logger  (DVL),  and  pressure  sensors.  The 
Ultra  Short  Base  Line  (USBL)  and  Global  Positioning  System  (GPS)  were  added  as  a  global 
vehicle  position  feedback  sensor  during  underwater  navigation  and  surface  navigation, 
respectively. 

However,  at  the  end  of  the  Phase  III- A,  it  was  evident  how  the  accuracy  and  precision  of 
this  sensor  system  was  insufficient,  in  particular  conditions,  during  the  manipulation  tasks. 
Thus  it  was  necessary,  during  the  phase  III-B,  to  introduce  a  more  reliable  Inertial 
Navigation  System  aimed  to  produce  the  position  data  with  the  reliability  necessary  to  the 
autonomous  intervention. 

This  important  change,  together  with  a  complete  re-design  of  the  navigation  controller, 
allowed  the  SAUVIM  vehicle  to  successfully  perform  in  compliance  with  the  precision, 
accuracy  and  stability  requirements  of  our  manipulation  task. 

Another  important  upgrade  of  the  Phase  III-B,  aimed  to  improve  the  coordinate  motion 
between  the  arm  and  the  vehicle,  was  the  standardization  of  all  the  communication 
protocols.  This  was  accomplished  with  the  extension  of  the  xBus  protocol,  once  dedicate  to 
the  arm  subsystem  only,  to  the  entire  vehicle.  xBus  showed  a  great  flexibility  in  handling 
every  kind  of  communication  (data,  program  code,  messages..  )  and  thus  it  was  chosen  as 
the  SAUVIM  standard. 

During  final  phase  (III-C)  the  navigation  controller  has  been  further  upgraded  with  very 
important  enhanced  feature  like  on-line  identification  of  the  center  of  buoyancy.  This  is  one 
of  the  most  important  capabilities  of  the  system  that  allowed  high  hover  stability  during 
manipulation  and  optimized  power  consumption.  The  Extended  Kalman  Filter  employed  in 
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the  identification  needs  a  relatively  precise  knowledge  of  the  thrust  force.  This  has  been 
subject  of  another  important  research  for  mapping  the  thrusters  actual  inputs  and  status  to 
the  actual  force  developed. 


Real-time  center  of  buoyancy  identification  of  underwater  vehicles  for  optimal 
positioning  in  autonomous  intervention 

This  research  addresses  the  problem  of  optimal  positioning  for  an  intervention  AUV, 
minimizing  the  energy  consumption  and  improving  the  stability  in  orientation.  During  a 
generic  intervention  task,  the  vehicle  is  generally  required  to  maintain  a  hovering 
configuration,  thus  requiring  a  6  DOF  control  of  the  vehicle  positioning.  The  choice  of  roll 
and  pitch,  if  done  arbitrarily,  can  severely  impact  the  power  efficiency  of  the  vehicle, 
especially  in  heavy  systems,  since  the  center  of  buoyancy  may  not  be  necessarily  aligned 
over  the  center  of  mass.  This  approach  uses  an  Extended  Kalman  Filter  to  identify  the 
location  of  the  center  of  buoyancy  w.r.t.  the  center  of  mass,  thus  allowing  to  compute  the 
working  orientation  that  maintains  the  COB  vertically  aligned  above  the  COM.  The  EKF  is 
implemented  online  and  hence  is  able  to  detect  movement  of  the  COB  due  for  example  to 
ballast  operations.  This  algorithm  has  been  firstly  implemented  in  simulation  and  then 
successfully  validated  with  the  SAUVIM  autonomous  underwater  vehicle.  With  its  weight 
of  about  4  tons,  this  testbed  is  an  optimal  platform  for  validating  the  precision  of  the  filter, 
since  a  very  small  variation  of  the  target  pitch  and  roll  results  in  a  large  restoring  torque. 

Introduction 

Underwater  intervention  operations  are  ordinarily  executed  using  manned  submersibles  or 
Remotely  Operated  Vehicles  in  tele-operation  mode.  Autonomous  Underwater  Vehicles  are 
generally  employed  in  survey  missions,  and  only  few  AUVs  are  today  equipped  with 
manipulators.  In  fact,  the  low  bandwidth  and  significant  time  delay  inherent  in  acoustic 
subsea  communications  represent  a  considerable  obstacle  to  remotely  operate  a 
manipulation  system,  making  it  impossible  for  remote  controllers  to  react  to  problems  in  a 
timely  manner. 

Nevertheless,  robots  for  autonomous  underwater  intervention  would  pave  the  way  for  a 
different  range  of  new  operations,  such  as  deep-ocean  and  under-ice  exploration,  tasks  in 
hazardous  areas,  in  natural  or  man-made  disastrous  regions,  automated  searches, 
surveillance  missions,  to  name  a  few. 

The  key  technology  in  underwater  intervention  performed  with  autonomous  vehicles  is 
autonomous  manipulation.  This  is  a  challenging  technology  milestone,  which  refers  to  the 
capability  of  a  robot  system  that  performs  intervention  tasks  requiring  physical  contacts 
with  unstructured  environments  without  continuous  human  supervision.  Autonomous 
manipulation  systems,  unlike  teleoperated  manipulation  systems  that  are  controlled  by 
human  operators  with  the  aid  of  visual  and  other  sensory  feedback,  must  be  capable  of 
assessing  a  situation,  including  self-calibration  based  on  sensory  information,  and  executing 
or  revising  a  course  of  manipulating  action  without  continuous  human  intervention. 
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During  a  generic  autonomous  manipulation  task  the  vehicle  controller  has  the  main 
responsibility  of  maintaining  the  vehicle  in  the  necessary  configuration.  For  example,  often 
the  vehicle  has  to  be  actively  stabilized  in  a  hovering  configuration,  while  the  manipulator 
performs  its  task.  Among  the  hydrodynamic  effects  acting  on  a  rigid  body  moving  in  a 
fluid,  the  restoring  generalized  forces  (gravity  plus  buoyancy)  and  the  ocean  current  are  of 
major  concern  in  designing  a  motion  control  law  for  intervention  AUVs.  In  literature, 
several  works  have  been  presented  in  order  to  assess  the  problem  of  6  DOF  control  of  AUVs 
(see  for  example  [1],  [2],  [3],  [4],  [5],  [6]).  An  analysis  of  the  capacity  of  the  above  works  to 
compensate  for  the  persistent  dynamic  effects,  e.g.,  the  restoring  forces  and  the  ocean 
current,  has  been  presented  in  [7]. 

In  general,  one  of  the  main  problem  is  the  lack  of  knowledge  of  the  restoring-related 
dynamic  parameters,  especially  in  case  of  heavy  vehicles.  In  particular,  the  location  of  the 
center  of  buoyancy  with  respect  to  the  center  of  mass  plays  a  fundamental  role  in  the 
performance  of  the  dynamic  control.  The  importance  of  its  knowledge  is  also  related  to  the 
problem  of  power  optimization,  since  very  often  a  working  orientation  of  the  vehicle  does 
not  have  strict  constrains,  being  the  manipulator  more  capable  of  realizing  specific 
orientation  than  the  vehicle.  Thus  a  working  vehicle  orientation  could  be  simply  chosen  as 
the  one  that  aligns  the  COB  above  the  COM,  minimizing  the  power  requirement  for 
maintaining  the  vehicle  in  hovering. 

In  this  research  we  present  a  methodology  for  identifying  the  relative  position  of  the  COB 
with  respect  to  the  center  of  mass.  This  approach  is  based  on  the  use  of  an  extended  Kalman 
filter  and,  being  suitable  of  real-time  implementation,  it  can  easily  adapt  to  any  change  in 
the  vehicle  configuration  (i.e.  ballast  operations  or  manipulator  dynamics). 

The  algorithm  has  been  successfully  tested  firstly  in  a  simulation  environment  and  then 
with  the  SAUVIM  underwater  vehicle,  in  both  cases  with  a  model-based  dynamic  control. 
Results  here  presented  have  been  proven  to  be  extremely  accurate,  and  thus  allowing  to 
optimize  the  hovering  of  the  SAUVIM  vehicle  during  its  manipulation  tasks. 

DYNAMICS  OF  THE  UNDERWATER  VEHICLE-MANIPULATOR  SYSTEM 


In  our  study  the  dynamics  has  been  modeled  with  particular  care  to  the  further  extension  of 
the  global  vehicle-manipulator  system  dynamics.  We  used  the  Lagrange  equation  for  quasi¬ 
coordinates,  since  this  approach  allows  an  easy  generalization  of  multibody  systems  with 
joints  not  limited  to  be  only  one  degree  of  freedom.  With  this  choice,  the  UVM  system 
becomes  a  linear  chain  of  simple  joints  with  the  first  one  being  of  6  DOF  (free  body).  In  this 
paper  we  are  not  considering  the  dynamics  of  the  manipulator,  since  we  assume  that  it  is  in 
its  parked  position  and  part  of  the  vehicle  dynamics.  With  this  assumption,  the  Lagrange 
equation  for  the  quasi-coordinates  becomes  (  [11],  [12],  [13]): 

^(q)p  +  5(q,p)p  =  FV  (1.1) 

where,  in  general: 
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•  [^4(q)]  is  the  inertia  matrix  of  the  structure  (comprehensive  of  the  added  mass  and 

added  inertia); 


•  [  B  ]  is  the  is  the  matrix  of  Coriolis  and  centrifugal  forces  (also  comprehensive  of  the 
added  mass  and  added  inertia); 

•  [  J^(q)]  is  the  transformation  matrix  between  the  p  space  and  the  q  space,  such  as 
q  =  Fp; 

•  [  VT /ue  ]  is  the  projection  in  the  space  of  the  joint  velocities  (i.e.  body  axis  in  our  case  ) 
of  the  external  generalized  forces. 

Let  j(q,,P)  be  the  jacobian  of  the  structure,  such  as: 


A--J(q,P)p  (1.2) 

where  X  is  the  generalized  velocity  (in  the  main  frame)  of  the  point  P  .  With  this 
assumption,  the  projection  in  the  space  of  the  joint  velocities  of  a  generalized  extern  action 

Wp  =  [MP  Fp  ]  is  given  by: 

fiP=JT(q,P)W  (1.3) 


In  modeling  our  vehicle,  since  we  consider  the  vehicle  stationary  in  a  hovering 
configuration,  the  extern  action  is  composed  of  the  restoring  torque,  linear  dumping  actions 
and  thruster  forces.  This  lead  to  the  final  form  of  the  (1.1): 

/i(q)p  +  Z?(q,p)p  =  Dp  +  TCMr  +  6r  (1.4) 

where  D  is  a  diagonal  matrix  associated  with  the  linear  damping  term  of  the  drag  force, 
TCM  is  the  thruster  control  matrix  and  9r  is  the  generalized  restoring  action,  given  by: 


dr=JT(q,CB) 
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(1.5) 
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Equation  (1.4)  implicitly  describes  the  evolution  of  the  state  vector  through  time.  It  can  be 
solved  with  respect  to  the  derivative  of  the  quasivelocity  vector  as: 

p  =  A  _1(-5p  +  Dp  +  TCM  t  +  )  (1.6) 

Then,  in  order  to  complete  the  state  space  equations  of  the  Kalman  filter,  we  can  add  4  more 
equations  describing  the  evolution  of  the  three  coordinates  of  the  center  of  buoyancy  and 
the  buoyancy  force.  Since  the  ballast  movements  are  supposed  much  slower  than  the 
dynamics  of  the  vehicle,  their  evolution  is  simply  given  by: 


4  =  o 
%  =  o 

4  =° 
Fb=  0 


(1.7) 


being  cbx ,  cby  and  chz  the  coordinates  of  the  center  of  buoyancy  with  respect  to  the  center  of 
mass,  and  Fb  the  resultant  of  the  buoyancy  force  (applied  to  the  COB). 

We  also  need  to  add  the  evolution  of  the  generalized  position  of  the  vehicle.  This  is  easily 
done  by  integrating  the  rotation  matrix: 


r  o  -p^  p 2i 


0  /?  — 0 

C/77  Cm 
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p3 


0  -pi 


-p2  pi  0 


and  the  linear  acceleration: 


°cm=cmR[P4  Ps  Pel 


(1.8) 


(1.9) 


Process 

The  complete  non-linear  process  can  be  described  putting  together  Eqs.  (1.6),  (1.7),  (1.8)  and 
(1.9),  and  adding  the  opportune  noise.  Since  several  parameters  are  known  approximatively, 
we  can  consider  adding  a  noise  variable  wt  to  each  one  of  them.  In  our  case  we  have  added 
noise  in  the  following  terms: 

Added  mass: 

^t=mt+wmt 


Added  inertia: 
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Center  of  mass  location  (for  the  computation  of  the  TCM : 

T 

Cm=cm  +  [wcmx,wcmy,wcmz^ 

Thrust  vector  (considering  that  our  vehicle  has  8  thrusters): 

r -r  +  [wrt,.,w,8J 

Damping  coefficients: 

D  =  diag(k,  +  wkd  ) 

With  this  assumptions,  after  discretizing  the  system  equations  (1.6)  and,  (1.7)  the  non-linear 
stochastic  difference  equation  of  our  extended  Kalman  filter  becomes: 


*k  = 
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(«.X 
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(1.10) 


where  AT  is  the  sample  time.  Note  that  many  noise  variables  are  encapsulated  within  A , 

B ,  TCM  and  r  .  The  last,  vector  of  the  thrust  forces,  is  the  process  input.  The  dimension  on 
the  state  vector  is  thus  22. 


Measure 

The  equations  of  the  measure,  in  our  system,  are  way  more  complicated  than  the  process 
evolution.  This  is  due  to  the  fact  that  the  Inertial  Navigation  system  (PHINS)  is  physically 
mounted  with  an  offset  (in  translation  and  rotation)  w.r.t.  the  center  of  mass  (see  Figure). 
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The  PHINS  is  basically  composed  by  another  extended  Kalman  filter  which  fuses  together 
sensor  data  from  fiber  optical  gyroscope,  accelerometers,  velocity  sensor  (DVL),  position 
sensor  (GPS)  and  a  depth  probe.  It  provides  in  output  the  following  information: 

The  PHINS  is  basically  composed  by  another  extended  Kalman  filter  which  fuses  together 
sensor  data  from  fiber  optical  gyroscope,  accelerometers,  velocity  sensor  (DVL),  position 
sensor  (GPS)  and  a  depth  probe.  It  provides  in  output  the  following  information: 

The  PHINS  is  basically  composed  by  another  extended  Kalman  filter  which  fuses  together 
sensor  data  from  fiber  optical  gyroscope,  accelerometers,  velocity  sensor  (DVL),  position 
sensor  (GPS)  and  a  depth  probe.  It  provides  in  output  the  following  information: 


The  PHINS  is  basically  composed  by  another  extended  Kalman  filter  which  fuses  together 
sensor  data  from  fiber  optical  gyroscope,  accelerometers,  velocity  sensor  (DVL),  position 
sensor  (GPS)  and  a  depth  probe.  It  provides  in  output  the  following  information: 

•  °PT  generalized  position  of  the  PHINS  w.r.t.  the  main  frame  <0>; 

•  °vph  is  the  linear  velocity  of  the  PHINS  projected  the  main  frame  <0>; 

•  P  coPh  is  the  angular  velocity  of  the  PHINS  projected  the  PHINS  frame  <Ph>; 

•  P  aph  is  the  linear  acceleration  the  PHINS  projected  the  PHINS  frame  <Ph>; 


The  above  outputs  represent  our  measure.  In  order  to  integrate  them  in  our  EKF  it  is 
necessary  to  express  the  quasivelicity  vector  and  its  derivative  in  term  of  the  PHINS 
outputs. 
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The  transformation  matrix  °PT  of  the  PHINS  frame  w.r.t.  the  main  frame  can  be  easily 
expressed  in  function  of  the  transformation  matrix  of  the  center  of  mass: 


0  rri  _ 0 

P1  Cm 


TCmT 


(1.11) 


which  is  function  of  the  state  variables  °Cm  R ,  cm  and  of  the  known  generalized  offset  pm  T 

(see  previous  Figure).  The  reorganization  of  the  transformation  matrix  T  into  a  12  element 
vector  represents  the  first  set  of  measure  equations. 

The  linear  velocity  of  the  PHINS,  projected  the  main  frame  <0>,  can  be  computed  from  the 
quasivelocity  vectors  making  use  of  the  linear  part  of  the  jacobian  (i.e.  the  bottom-half  of  the 
matrix)  of  Eq.(1.2): 


(1.12) 


where  rp  is  the  location  of  the  origin  PHINS  frame  <Ph>  w.r.t.  the  main  frame  <0>.  Details 
on  the  computation  of  the  Jacobian  here  used  can  be  found  in  [11]. 

The  angular  velocity  of  the  PHINS,  projected  the  main  frame  <0>,  is  simply  given  by: 

no>n  -rci  Rcm°>c„  -£  *[a  ft  ft  I  (1.13) 

Finally,  it  is  necessary  to  express  the  linear  acceleration  of  the  PHINS  in  term  of  derivative  of 
the  quasivelocity  vector.  This  can  be  done  considering  the  relation  between  the  derivative  of 
the  configuration  vector  q  and  the  quasivelocity  p : 


d_ 

dt 


dV  T,  do 

= - p  +  F  — 

dt  dt 


(1.14) 


The  last  three  elements  of  the  vector  (1.14)  represent  the  linear  acceleration  °aCm  of  the 
center  of  mass  projected  in  the  main  frame  <0>.  This  is  related  to  the  linear  acceleration  °a/J/? 
of  the  PHINS  frame,  projected  in  the  main  frame,  by  the  following  relation: 

°a™  =°  aan  +°  vcm  A°  rp  +L  Rfo  Pi  Pi  1  A°  rP  (1-15) 

Note  that  the  quantity  —  can  be  easily  computed  from  the  first  6  state  equations  (1.6). 

dt 


Finally,  combining  together  equations  (1.11),  (1.12),  (1.13)  and  (1.15)  we  obtain  a  set  of  21 
equations  which  represent  the  measure  equations  of  the  extended  Kalman  filter: 

z*  =  [(o^J +v*7*  (Ov^X+v*  (Ph(oph\+vwk  (Oa  Ph\+vak^  (1.16) 
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The  4  vectors  V  represent  the  measure  noise,  associated  with  the  PHINS  unit. 


Optimal  configuration  for  hovering 

The  importance  of  the  knowledge  of  the  center  of  buoyancy  is  also  related  to  the  problem  of 
power  optimization.  During  an  autonomous  manipulation  task,  the  working  orientation  of 
the  vehicle  does  not  have  strict  constrains,  being  the  manipulator  more  capable  of  realizing 
specific  orientation  than  the  vehicle.  Thus  a  working  vehicle  orientation  could  be  simply 
chosen  as  the  one  which  aligns  the  COB  above  the  COM,  minimizing  the  power 
requirement  for  maintaining  the  vehicle  in  hovering.  The  equilibrium  target  rotation  is  thus 
computed  with  the  following  considerations. 

Let  the  rotation  matrix  of  the  equilibrium  configuration  be 


o 

E 


0 


h 


(1.17) 


To  reach  our  goal,  the  axis  °ke  is  chosen  to  be  the  parallel  to  the  COM-COB  segment: 


o 


K 


(1.18) 


The  axis  °ie  is  instead  chosen  as  the  one  orthogonal  to  the  plane  formed  by  °ke  and  the  axis 


[0  1  0  J : 


[0  1  oT*"k, 

*'[01  0jA"k, 


(1.19) 


Finally,  the  0  je  axis  is  simply  computed  as  the  cross  product: 


o 


j  =°k 

Je  < 


a°  i 


(1.20) 


The  matrix  (1.17)  so  formed  is  then  transformed  in  roll,  pitch  and  yaw  angles  and  the  first 
two  replace  the  ones  target  orientation  for  the  navigation  controller. 

Implementation 

The  complex  nature  of  our  EKF  has  been  the  main  obstacle  to  its  practical  implementation. 
In  SAUVIM,  this  has  been  overcome  by  means  of  automated  tools  for  code  generation.  The 
computation  of  the  partial  derivatives  of  the  process  measure,  necessary  to  the 
implementation  of  the  extended  Kalman  filter,  has  been  done  using  symbolic  computation 
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packages.  The  symbolic  processor  has  then  been  used  for  generating  the  5,000+  lines  of  C++ 
code  necessary  to  import  the  EKF  in  our  hardware  system. 

Despite  the  complexity  and  length,  with  the  optimization  introduced  by  the  code  generation 
process,  the  release  version  of  our  EKF  is  able  to  run  with  a  sample  time  of  about  50ms, 
hence  perfectly  suitable  to  an  online  implementation  in  the  vast  majority  of  navigation 
controllers. 

Simulative  results 

The  first  results  of  the  EKF  for  COB  identification  have  been  validated  using  the  SAUVIM 
simulator.  The  simulator  implements  the  Lagrange  equation  for  quasicoordinates  (1.4), 
inclusive  of  the  drag,  restoring  force  and  added  mass. 

The  first  experiment  consisted  in  performing  simple  open-loop  oscillation  around  the  three 
center  of  mass  axis,  in  order  to  assess  the  performance  of  the  EKF.  Results  are  shown  in  the 
following  figures. 
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Here,  the  three  coordinates  of  the  center  of  buoyancy  successfully  converge  toward  the 
simulated  values  of  the  COB  (respectively  -10mm,  -5mm  and  200mm).  The  accuracy,  in  this 
simulative  case,  is  in  the  order  of  fraction  of  millimeter. 


Experimental  results 

The  previous  results  were  experimentally  confirmed  with  the  SAUVIM  vehicle.  In  this  case 
the  vehicle  is  set  to  maintain  an  hovering  position,  with  the  roll  and  pitch  chosen  in  order  to 
align  the  center  of  buoyancy  over  the  center  of  mass.  In  this  experiment  the  controller  is  a  6 
DOF  model-based  [7],  re-adapted  to  the  quasivelocity  formulation  ([11],  [13]). 

The  following  plots  show  the  output  of  the  Kalman  filter  subsequent  to  three  different 
changes  of  the  COB  position,  obtained  by  re-distributing  the  air  in  the  ballasts. 
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The  proof  that  the  COB  lies  above  the  COM  is  that  the  global  module  of  the  thrust  vector 
converges  toward  zero  (see  figure).  As  a  matter  of  fact,  even  a  small  fraction  of  degree  in  the 
target  orientation  would  generate  a  compensation  thrust  of  several  kilograms,  given  the 
mass  of  3,660kg  and  90mm  of  distance  between  the  COB  and  COM  of  the  vehicle. 

The  condition,  in  this  experiment,  were  the  same  of  the  simulative  case,  and  the  model 
based  controller  was  set  to  maintain  a  hovering  position  over  a  submerged  reference  target. 
The  global  buoyancy  of  SAUVIM,  in  this  case,  was  set  to  neutral. 


In  particular,  right  after  the  initialization,  the  estimated  value  of  the  COB  was  done  with  a 
larger  error  and  the  thrusters  had  to  compensate  the  restoring  force  in  order  to  maintain  the 
hovering  position.  Successively,  the  EKF  converges  toward  the  COB  location  with  a  sub¬ 
millimeter  accuracy,  proven  by  the  convergence  toward  zero  of  the  thrust  force. 
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Active  Feedback  Thruster  System  (AFTS) 

Project  Leader (s):  Mr.  Aaron  Hanai 

Personnel:  Mr.  Kaikala  Rosa 


Objectives 


Since  one  of  the  primary  goals  for  the  vehicle  is  underwater  manipulation,  the  thruster 
subsystem  must  be  accurate  enough  to  maintain  robust  hovering  of  the  vehicle.  This  has 
required  experimental  analysis  and  tuning  of  the  hardware  and  engineering  design  into  the 
performance  of  the  software  feedback  control  scheme.  The  objectives  of  this  system  include: 

•  An  energy  efficient  distribution  of  forces  among  the  8  vehicle  thrusters  using  an 
analytical  approach  (as  opposed  to  heuristics). 

•  A  closed-loop  thruster  control  design  based  on  feedback  from  the  motor 
controllers. 

•  A  software  supervisor  to  prevent  errors  when  the  reference  thrust  exceeds  the 
physical  limits  of  the  hardware  due  to  voltage  sag  in  the  source  batteries. 

•  A  model-based  thrust  estimator  that  is  robust  to  unfavorable  water  conditions  in 
which  cavitation  may  occur. 


Current  Status  (Tasks  Completed  during  10/25/2006  -  10/30/2009) 


1.  Thruster  force  allocation: 

•  Definition  of  a  variable  thruster  configuration  matrix  mapping  between  the  thruster 
forces  and  the  body-fixed  vehicle  forces/ torques 

•  Solution  of  the  thruster  configuration  matrix  via  weighted  pseudoinverse 

2.  Saturation  Guard: 

•  Separation  and  isolation  of  the  linear  and  angular  thrust  errors 

•  Modeling  of  thrust  loss  due  to  battery  voltage  sag 

3.  Thruster  modeling: 

•  Experimental  analysis  of  the  functional  relationships  between  thruster  input 
reference  voltage,  measured  current,  measured  velocity,  and  output  thrust 

•  Development  of  model-based  thrust  approximation  functions 

4.  Cavitation  tolerance: 

•  Experimental  observation  of  the  effects  of  cavitation  on  thruster  performance 

•  Development  of  a  model-based  fault  tolerant  thrust  estimation  function  (robust  to 
cavitation) 

•  Development  of  a  fault  accommodating  thruster  system  that  scales  the  thruster 
configuration  matrix  based  on  the  error  estimation 
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Automatic  Fault- Accommodating  Thrust  Redistribution  for  a  Redundant  AUV 

This  research  reports  on  the  development  of  an  automatic  thrust  redistribution  algorithm 
for  a  redundant  autonomous  underwater  vehicle  (AUV)  in  the  case  of  thruster  faults.  Under 
favorable  conditions,  the  transformation  of  the  thruster  force  vector  to  the  generalized  body- 
fixed  force  vector  can  be  computed  in  a  least-squares  manner,  which  provides  the  minimum 
norm  solution.  In  the  case  of  some  type  of  thruster  fault,  the  redundancy  in  the  system 
allows  for  seamless  redistribution  of  the  thruster  forces  while  maintaining  the  desired 
vehicle  motion.  An  accurate,  fault-detecting  thrust  estimator  has  been  previously 
developed  that  is  robust  to  changes  in  source  power  and  propeller  ventilation.  Such  an 
estimator  is  necessary  in  order  to  quantify  the  health  of  the  thrusters,  which  can  then  be 
used  to  weigh  the  contribution  of  each  thruster  to  the  vehicle  motion  in  a  continuous 
manner.  An  example  of  the  thruster  health  index  in  its  simplest  form  is  the  approximated 
thrust  divided  by  the  reference  thrust,  passed  through  a  moving  average  filter,  and 
normalized  over  all  thrusters.  This  index  can  effectively  scale  back  the  contribution  of  any 
faulty  thruster  to  the  body-fixed,  vehicle  force  vector  solution. 


1.  Introduction 

The  primary  motivation  of  this  paper  is  to  evolve  the  station-keeping  ability  of  an  AUV  for 
the  purpose  of  autonomous  manipulation,  as  implemented  for  intervention  tasks.  Fine 
motion  control  of  the  vehicle  is  necessary  since  its  motion  is  translated  to  the  end-effector. 
Furthermore,  the  unpredictable  nature  of  subsea  conditions  during  a  mission  requires  a 
fault-accommodating  system  design  that  is  robust  to  unfavorable  water  conditions  or 
internal  subsystem  failures.  For  example,  the  propellers  may  become  entangled  or 
damaged,  ventilation  may  occur  near  the  water  surface,  or  there  may  be  a  reduction  in 
source  power  to  the  thrusters. 

To  design  an  appropriate  thrust  estimator,  the  propeller  physics  may  be  considered  in  order 
to  develop  a  set  of  single  or  multiple  state  mathematical  models  for  the  thrusters  [1-5]. 
However,  since  the  servo  motor  controllers  associated  with  the  thrusters  may  provide 
velocity  and  motor  current  draw  as  feedback,  the  data  may  be  compared  to  measured  thrust 
values  (by  load  cell)  in  order  to  develop  a  set  of  easily  and  practically  derived  thruster 
models  [6].  This  was  the  method  employed  to  derive  the  existing  thruster  models,  although 
due  to  the  lack  of  measurement  equipment,  the  ambient  water  flow  velocity  [7-8]  was  not 
considered,  which  would  have  theoretically  improved  the  accuracy  of  the  models. 

A  fault-tolerant  thruster  system  design  can  be  divided  into  the  areas  of  fault  detection,  fault 
isolation,  and  fault  accommodation  [9].  Examples  of  fault  detection  and  isolation  include  a 
statistical  method  [10],  a  trained  diagnostic  observer  [11],  or  a  practically  derived,  model 
based  approach  [12].  Provided  a  successful  thruster  fault  detection  method,  the  desired 
vehicle  body-force  reference  can  be  achieved  by  distributing  the  thruster  forces  by  means  of 
a  weighted  least-norm  [13],  or  bounded  infinity-norm  solution  [14]. 

This  paper  focuses  on  the  details  of  innovative  fault  accommodation  in  particular.  The 
difference  between  a  thruster's  reference  value  and  its  corresponding  estimate  can  generate 
a  type  of  quantified  health  index.  This  value  can  subsequently  be  used  to  scale  back  the 
contribution  of  a  particular  thruster  to  the  overall  vehicle  motion  by  adjusting  the  weights  in 
the  mappings  between  the  vehicle-space  and  thruster-space  configurations.  Furthermore, 
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this  can  be  implemented  in  a  continuous  manner,  as  opposed  to  a  thruster  that  is  discretely 
deemed  either  on  or  off. 


Fig.  1  SAUVIM 


2.  Thrust  Distribution 

The  basic  thrust  distribution  algorithm  is  a  component  of  an  active  feedback  thruster  system 
that  is  implemented  and  tested  on  the  Semi- Autonomous  Underwater  Vehicle  for 
Intervention  Missions  (SAUVIM)  [15],  shown  in  Fig.  1.  Development  is  a  collaborative 
project  between  Marine  Autonomous  Systems  Engineering,  Inc.  (MASE,  Inc.),  the 
Autonomous  Systems  Laboratory  of  the  University  of  Hawaii's  College  of  Engineering,  and 
the  Naval  Undersea  Warfare  Center,  Rhode  Island.  The  vehicle  is  outfitted  with  a  seven 
degree-of-freedom  electromechanical  robotic  manipulator,  and  as  a  unified  system  is 
designed  for  intervention  tasks.  This  requires  fine  motion  control  for  hovering  and  is 
therefore  equipped  with  eight  brushless  DC  thrusters  for  geometric  redundancy. 

Vehicle  sensors  feed  kinematic  information  to  the  navigation  controller,  which  develops  a 
body-fixed  reference  force  vector  r,  which  is  derived  from  the  input  of  the  vehicle  equation 
of  motion  [16]: 

M(q)p  +  D(q,  p)p  +  g(q)  =  T  (1) 

where  p  is  the  linear  and  angular  velocities  in  the  body-fixed  frame  and  q  is  the  vehicle 
position  and  orientation  vector  in  the  earth-fixed  frame.  M  is  the  inertia  matrix,  including 
both  rigid  body  and  added  mass  terms.  D  is  the  matrix  of  dissipative  terms  including 
Coriolis  and  centripetal  effects,  as  well  as  hydrodynamic  damping.  The  vector  g  describes 
the  gravitational  and  buoyant  restoring  forces. 

As  applied  to  SAUVIM,  its  navigation  controller  solves  this  equation  of  motion  in  order  to 
generate  a  6-DOF  reference  vector  rref  from  the  kinematic  values  measured  by  the  sensors. 
This  vector  must  be  transformed  to  a  thruster  reference  vector  T  =  [Ti  •  •  •  Ts]  according  to  the 
geometry  in  Figs.  5  and  6,  where  thrusters  1  through  4  are  oriented  vertically,  5  and  6  are 
longitudinal,  and  thrusters  7  and  8  are  laterally  aligned. 

Defining  the  positions  and  orientations  Xi  and  n  of  thruster  i  respectively,  as  well  as  the 
vehicle  center  of  mass  C  =  [ Cx ,  Cy,  CJ,  then  by  geometry, 

T  =  KT  (2) 
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according  to  the  transformation  matrix 


[(x1-C)xr1  •••  (x8-C)xr8J 

At  this  point,  (2)  can  be  inverted  such  that 

T  =  K#r  (4) 

according  to  the  generalized  inverse  [17] 

K *  =  W~'Kt  ( KW~'Kt  )"'  (5) 


which  minimizes  the  error  norm  |  |  r  -  KT  |  |  of  (2).  Consistent  with  the  method  outlined  in 
[13],  the  diagonal  weight  matrix  W  can  be  defined  such  that 


w-1 


~wx  0  O' 
0  \  0 
0  0  w8 


(6) 


where  the  coefficients  (0  <  zvt  <  1)  can  each  be  set  to  some  value  based  on  the  reliability  of  the 
corresponding  thruster.  In  this  case  (zvt  =  1)  represents  complete  functionality,  and  (zut  =  0) 
represents  complete  thruster  failure. 


3.  Thrust  Estimation 

To  develop  the  thrust  estimators,  experiments  were  performed  to  measure  the  relationship 
between  the  control  input  voltage,  feedback  current,  feedback  velocity,  and  output  thrust 
(measured  via  load  cell).  These  relative  measurements  are  displayed  versus  time  in  Fig.  2. 
Two  independent,  single-input  thrust  approximation  functions  were  developed  with  respect 
to  the  measured  electric  current  draw,  and  the  measured  propeller  shaft  velocity.  The 
purpose  for  the  two  separate  functions  is  apparent  when  subsea  conditions  become 
unfavorable.  When  ventilation  occurs,  which  is  common  if  the  vehicle  is  at  the  surface,  the 
two  thrust  approximations  deviate  from  each  other.  This  difference  can  quantify  the  state  of 
thruster  fault.  The  first  half  of  the  time  interval  in  Fig.  3  represents  favorable  subsea 
conditions,  such  that  the  test  thruster  is  a  sufficient  distance  below  the  surface  of  the  water. 
In  this  case,  the  measured  thrust  follows  the  reference,  and  the  two  thrust  approximations 
agree  with  the  measured  value.  However,  in  the  second  half  of  Fig.  3,  the  thruster  is 
purposely  positioned  near  the  water  surface  in  order  to  promote  propeller  ventilation.  In 
this  case,  the  measured  thrust  no  longer  follows  the  reference,  and  the  velocity-based  thrust 
approximation  is  overestimated  relative  to  the  measured  value,  while  the  current-based 
approximation  is  also  overestimated,  but  to  a  lesser  extent.  This  relationship  was  exploited 
in  order  to  develop  a  new  thrust  approximation  that  accounts  for  ventilation  by  considering 
both  current  and  velocity  feedback  measurements  [12].  The  experiment  performed  as 
plotted  in  Fig.  4  is  identical  to  that  in  Fig.  3,  except  that  the  new  thrust  approximation 
function  is  robust  to  ventilation  effects. 
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4.  Thruster  Fault  Accommodation 


4.1  Thruster  Weights 

In  its  simplest  application,  the  thrust  estimate  Test  based  on  the  feedback  current  I  and 
feedback  velocity  Q  can  be  combined  with  the  reference  thrust  Tref  to  generate  a  quantified 
health  index  value  per  thruster  according  to 


w  = 


|rJ>o,  w(/-o)-i  (7) 


Note  that  the  values  for  w  are  initialized  to  1  at  the  start  of  a  mission,  where  time  t  =  0.  Also, 
if  Tref  =  0  for  a  particular  time  step,  then  w  does  not  need  to  be  updated,  but  rather  maintains 
its  previous  computed  value  (zero-order  hold).  During  favorable  conditions,  the  value  for  w 
is  unity,  and  scales  down  toward  zero  as  the  thrust  approximation  deviates  from  the 
reference  due  to  some  form  of  internal  fault  or  external  environmental  effect.  The  health 
indices  defined  in  (7)  can  be  input  as  the  coefficients  of  the  weight  matrix  from  (6). 
Effectively,  the  contribution  of  any  thruster  that  is  experiencing  ventilation  or  any  other 
fault  condition  will  be  scaled  down  relative  to  the  other  components  of  the  thrust  vector  in  a 
continuous  manner. 


4.2  Thruster  Weight  Smoothing 

In  order  to  smooth  the  action  of  the  thruster  weights,  a  basic  signal  processing  filter  was 
employed.  The  simple  moving  average  for  the  thruster  weight  wt  at  time  sample  t  is 


w 


0)=-y  w 


w  (/  =  ())  =  1  (8) 


which  provides  the  unweighted  mean  of  the  previous  n  data  points.  Experiments  with 
SAUVIM  thrusters  have  proven  successful  with  a  value  of  n  that  yields  about  a  10  second 
moving  average  data  window. 


Fig.  2  Control  input  voltage,  feedback  current,  feedback  propeller  velocity,  and  measured  thrust 

versus  time 
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Fig.  3.  Thrust  versus  time  with  unfavorable  conditions  (ventilation  occurs)  during  second  half  of 

experiment. 


Fig.  4  Reference,  measured,  and  ventilation-robust  thrust  approximation  during  unfavorable 

conditions 

From  the  definition  of  the  weight  matrix  in  (6),  the  values  of  the  time  averaged  coefficients 
must  be  constrained  so  that 
0  <  w(?)  <  1  (9) 

However,  any  instantaneous  measure  of  the  thruster  health  index  wt  is  allowed  to  exceed  unity 
0  <  w  <  w  ,  w  >1  (10) 

which  occurs  when  the  estimated  thrust  is  greater  than  the  reference,  and  indicates  that  the  thruster 
has  recovered  from  its  fault  condition.  This  can  happen  when  thruster  ventilation  initially  occurs  at 
the  water  surface,  but  then  the  vehicle  submerges  to  a  depth  sufficient  to  halt  the  ventilation. 
Conversely,  if  the  instantaneous  measure  wt  is  not  allowed  to  exceed  unity,  then  the  time  averaged 
value  recovers  its  weight  slowly  because  of  the  interval  of  the  moving  average,  and  furthermore 
would  be  unable  to  fully  recover  to  complete  functionality  where  w(t)=  1. 

It  is  also  necessary  that  the  time  averaged  thruster  health  index  remain  strictly  above  zero 
0  <  wmin  <  w (/)  <  1  (11) 

This  is  important  because  once  a  weight  value  w  goes  to  zero,  then  its  corresponding  thruster 
reference  also  goes  to  zero.  The  consequence  is  that  the  coefficient  w  would  no  longer  get  updated, 
and  the  thruster  reference  would  subsequently  be  trapped  at  zero. 


71 


4.3  Thruster  Weight  Normalization 

Furthermore,  it  is  necessary  to  normalize  the  time  averaged  thruster  health  indices  so  that  the  weight 
matrix  receives  a  relative  indication  of  the  thruster  faults  instead  of  an  absolute  measure.  Collecting 
the  time  averaged  health  indices  at  time  t  for  all  n  thrusters  into  a  single  vector,  then  the  normalization 
yields 

[w,  0)  •••  w  (0]  /  IX„(0](i2) 

This  ensures  that  at  least  one  thruster  weight  is  equal  to  1.  It  has  been  observed  on  SAUVIM  that  it  is 
possible  for  all  eight  of  the  thrusters  to  experience  some  form  of  estimated  inaccuracy.  In  this  case, 
without  the  implementation  of  the  thruster  weight  normalization,  all  n  of  the  weight  coefficients  may 
be  reduced,  and  the  reference  vector  Tref  will  have  a  large  component  in  the  nullspace  of  the  weighted 
pseudoinverse  solution  from  (4)  and  (5).  This  translates  empirically  to  a  waste  of  thruster  energy. 

5.  Testing 

In  order  to  illustrate  the  function  of  the  automatic  thrust  redistribution  algorithm,  a  simple  planar  simulation  was 
analyzed.  The  reference  thrust  input  is  set  as  a  pure  surge  motion,  linearly  ramped  up  to  200N  over  10  seconds. 

r ref  =  [surge  sway  yaw]  =  [  x(  t)  0  0] 

, ,  f 20? ,  t <.\o  (13) 

[200  ,  t  >  10 

The  starboard  longitudinal  thruster  is  purposely  limited  to  an  output  of  30N  to  simulate  a 
reduction  in  its  health  index.  Figs.  5  and  6  illustrate  snapshots  of  the  thruster  outputs  at 
t  =  7  and  t  =  50  seconds  into  the  simulation  respectively.  The  larger  arrow  outlines 
represent  the  reference  thrust  values  Tref  computed  from  the  weighted  pseudoinverse 
solution  of  the  rref  input  from  above.  The  smaller  arrows  within  the  outlines  represent  the 
estimated  thrust  values.  Fig.  7  plots  the  reference  inputs  and  estimated  outputs  for  the 
thrusters.  Figs.  8  and  9  show  the  thruster  errors  and  body  force  errors  respectively,  and 
Fig.  10  plots  the  thruster  weights. 

During  the  first  3  seconds  of  the  test,  the  thrust  estimates  track  the  reference  (Fig.  7),  but  the 
simulated  fault  for  thruster  6  limits  its  output.  This  results  in  an  error  in  its  output  (Fig.  8), 
leading  to  its  time  averaged  reduction  in  weight  (Fig.  10).  Due  to  its  decreasing  weight 
index,  the  reference  for  thruster  6  continues  to  reduce  until  its  error  stabilizes  its  weight.  At 
this  point,  all  of  the  thruster  errors  (Fig.  8)  are  reduced,  and  therefore,  so  are  the  body  force 
errors  (Fig.  9).  A  direct  comparison  of  Figs.  5  and  6  should  clearly  illustrate  the  effect  of  the 
thrust  redistribution.  The  former  case  shows  a  minimum-norm  reference  that  is  unable  to 
be  realized  due  to  a  thruster  fault.  The  latter  case  includes  the  addition  of  a  homogeneous 
solution  reference  derived  from  the  weighted  pseudoinverse  solution,  which  uses  more 
overall  thruster  power,  but  automatically  accommodates  the  fault  condition. 


6.  Conclusion 

An  automatic  fault-accommodating  thrust  redistribution  algorithm  has  been  successfully 
implemented  by  quantifying  the  health  of  the  thrusters,  and  using  the  resulting  values  to 
continuously  scale  the  contribution  of  each  thruster  to  the  overall  vehicle  motion  by  means 
of  a  weighted  pseudoinverse  solution.  The  thruster  weights  are  smoothed  by  means  of  a 
simple  moving  average,  and  then  normalized.  This  procedure,  combined  with  the  careful 
consideration  of  the  upper  and  lower  limits  of  the  weights,  ensures  that  the  thruster 
references  are  continuous  and  can  recover  from  fault  conditions.  The  result  is  a  thrust 
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distribution  that  automatically  reconfigures  itself  as  an  encapsulated  subsystem,  at  a  lower 
level  than  the  navigation  controller. 
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Fig.  5  Pure  surge  along  positive  x-axis,  with  reduced  output  on  thruster  6,  and  no  thrust  redistribution 


Fig.  6  Pure  surge  along  positive  x-axis,  with  reduced  output  on  thruster  6,  and  active  thrust 

redistribution 
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Fig.  7  Simulation:  Reference  inputs  and  estimated  outputs  for  SAUVIM  longitudinal  and  lateral 

thrusters 


Fig.  8  Thruster  errors 


Fig.  9  Body  force  errors 
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Localization  and  Navigation  (LN) 


Project  Leader (s): 
Personnel: 

Past  Project  Leader (s): 
Past  Personnel: 


Dr.  Giacomo  Marani 

Dr.  Son-Cheol  Yu,  Dr.  Giacomo  Marani 

Dr.  Son-Cheol  Yu,  Dr.  Junku  Yuh,  Dr.  Tae  Won  Kim,  Dr.  Song  K. 
Choi  &  Mr.  Michael  West 

Mr.  Kaikala  H.  Rosa,  Mr.  Scott  A.  Menor,  Mr.  Daniel  Shnidman  & 
Mr.  Mike  Hall 


Objectives 


Global  Localization  of  SAUVIM  in  all  the  different  condition  (on  the  surface  and 
underwater). 


Current  Status  (Tasks  Completed) 


The  task  has  been  completed  in  the  previous  phases.  Refer  to  previous  reports  for  its 
descriptions. 
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High-Level  Control  (HLC) 


Project  Leader (s): 
Personnel: 

Past  Project  Leader (s): 
Past  Personnel: 


Dr.  Giacomo  Marani 
Dr.  Giacomo  Marani 

Dr.  Tae  Won  Kim  ,  Dr.  Junku  Yuh,  Dr.  Kazuo  Sugihara  &  Dr.  Song 
K.  Choi 

Mr.  Side  Zhao,  Ms.  Jing  Nie  &  Mr.  Zhi  Yao 


Objectives 


HLC's  objective  is  to  develop  a  supervisory  control  module  that  will  minimize  human 
involvement  in  the  control  of  the  underwater  vehicle  and  its  manipulation  tasks. 


Current  Status  (Tasks  Completed) 


The  task  has  been  completed  in  the  previous  phases.  Refer  to  previous  reports  for  its 
descriptions. 
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Virtual  Environment  (VE) 

Project  Leader (s):  Dr.  Giacomo  Marani 

Personnel:  Dr.  Giacomo  Marani 

Past  Project  Leader (s):  Dr.  Song  K.  Choi,  Dr.  Kazuo  Sugihara,  Dr.  Stephen  Itoga  &  Mr. 

Scott  Menor 

Past  Personnel:  Mr.  Alexander  Nip,  Mr.  Zhenyu  Yang,  Mr.  Jiwen  Liu,  Mr.  Steve 

Timcho,  Ms.  Lori  Yokota,  Ms.  Jennifer  Saito,  Mr.  Brandon  Higa, 
Mr.  Xiandong  Su,  Mr.  Alberto  Brunete,  Ms.  Tammy  Yamauchi  & 
Mr.  Jeffery  P.  Yee 


Objectives 


The  VE  is  aimed  at  developing  a  supervisory  monitoring  system  for  SAUVIM  to  smoothly 
and  realistically  integrate  mapping  data  with  on-line  sensory  information  even  in  the  case  of 
low  bandwidth.  It  is  the  evolution  of  the  old  idea  of  the  Predictive  Virtual  Environment, 
described  in  the  previous  reports  of  SAUVIM,  into  a  more  advanced  system  collecting  also 
the  virtual  manipulator  and  the  SAUVIM  control  interface  through  direct  interaction  with 
the  virtual  environment. 


Current  Status  (Tasks  Completed) 


MarisGL  was,  during  the  Phase  II,  the  preliminary  version  of  the  virtual  environment 
targeted  to  the  MARIS  7080  Manipulator  and  making  use  of  a  standard  OpenGL  PC  video 
accelerator.  During  the  Phase  III-A  the  application  was  extended  in  order  to  introduce  the 
vehicle  model,  mainly  for  collision  avoidance  verification.  But  the  most  important  transition 
toward  the  global  virtual  environment  happened  in  the  current  Phase  III-B. 

Here,  the  name  of  the  application,  once  targeted  to  visualize  only  the  configuration  of  the 
arm,  has  been  changed  to  Sauvim  Explorer  (Figure  VE-1).  Sauvim  Explorer  collects  in  a 
unified  application  the  data  from  all  the  sensors  of  SAUVIM,  including  data  from  the 
DIDSON  that  can  be  overlaid  over  the  graphical  reconstruction  of  the  floor. 

In  the  final  phase  III-C  we  added  the  important  capability  of  data  recording  and  playback, 
thus  allowing  an  easy  analysis  and  post-process  of  all  the  mission  data. 
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amsaii 


Figure  VE-1:  Sauvim  Explorer 


It  also  hosts  the  remote  console  clients  for  both  the  Arm  Programming  Language  and  the 
Sauvim  Programming  Language  servers,  and  may  act  as  remote  control  (ROV  mode)  when 
a  sufficient  bandwidth  channel  is  present.  At  this  aim  Sauvim  Explorer  contains  software 
interface  with  several  input  device  hardware,  including  6  DOF  space  controllers. 

This  represents  an  enormous  step  forward  toward  the  unification  of  the  whole  system,  since 
it  required  a  huge  effort  on  the  standardization  of  the  communication  protocol  between 
every  module  of  SAUVIM  (sensors,  actuators,  controllers...).  With  this  modular  approach  it 
is  now  extremely  easy  to  add  further  sensor  modules  to  SAUVIM  and  add  their  input  and 
outputs  to  the  SE  application  with  a  minimal  effort. 

The  following  is  the  summary  of  the  major  key  points: 

•  Unified  interface  for  SAUVIM  and  MARIS  Manipulator 

•  Support  for  SPL  (Sauvim  Programming  Language)  and  APL  (Arm  Programming 
Language)  clients  in  the  same  console 
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•  Integration  of  the  DIDSON  interface 

•  Integration  of  the  altimeters 

•  Integration  of  the  pan-tilt  control 

Following  some  screenshots  of  the  actual  Virtual  Reality  interface. 


Figure  VE-2:  General  Interface 
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Figure  VE-3:  The  Generalized  Position  display 


Figure  VE-4:  Real-time  terrain  generator  (height  mapping)  for  the  virtual  reconstruction  of 
the  ocean  floor,  with  real-time  overlay  of  the  DIDSON  image 
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Figure  VE-5:  Support  for  6  DOF  motion  controller  devices,  for  an  alternate  driving  solution 
for  both  the  vehicle  and  manipulator  (in  case  of  teleoperation/  teleguidance) 


Figure  VE-6:  Real-time  link  with  the  Arm  subsystem 
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SAUVIM  Design  (SD) 

Project  Leader (s):  Dr.  Giacomo  Marani,  Dr.  Song  K.  Choi 

Past  Project  Leader (s):  Dr.  Curtis  S.  Ikehara,  Dr.  Junku  Yuh,  Dr.  Mehrdad  Ghasemi 

Nejhad,  Dr.  Gary  McMurtry,  Dr.  Pan-Mook  Lee,  Dr.  Farzad 
Masheyekhi,  Dr.  Gyoung  H.  Kim,  Mr.  Gus  Coutsourakis,  Mr. 
Oliver  T.  Easterday  &  Mr.  Michael  E.  West 

The  main  technical  development  of  the  SD  group  is  described  in  the  following  sections: 
Reliable,  Distributed  Control,  Mission  Sensor  Package,  Hydrodynamic  Drag  Coefficient 
Analysis,  Mechanical  Analysis  &  Fabrication  and  Mechanical-Electrical  Design.  Many  of  the 
developments  relative  to  the  SD  group  have  been  competed  in  the  previous  phases. 
However  the  Phase  III-B  has  seen  substantial  changes  in  the  Reliable,  Distributed  Control, 
here  described. 
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Reliable  Distributed  Control  (RDC) 


Project  Leader (s): 
Personnel: 

Past  Project  Leader (s): 
Past  Personnel: 


Dr.  Giacomo  Marani 
Dr.  Giacomo  Marani 

Dr.  Tae  Won  Kim,  Dr.  Pan-Mook  Lee,  Dr.  Curtis  S.  Ikehara,  Dr. 
Song  K.  Choi  &  Dr.  Gyoung  H.  Kim 

Mr.  Jang-Won  Lee,  Mr.  Michael  West,  Mr.  Tuan  M.  Hyunh,  Dr. 
Hyun  Taek  Choi,  Mr.  Alberto  Brunete  &  Mr.  Alexander  Nip 


Objectives 


The  objective  is  to  develop  a  reliable  &  efficient  computing  architecture  for  signal  and 
algorithmic  processes  of  the  entire  SAUVIM  system. 


Current  Status  (Tasks  Completed) 


The  task  has  been  completed  in  the  previous  phases.  Refer  to  previous  reports  for  its 
descriptions. 
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Mission  Package  Sensors  (MSP) 


Project  Leader (s): 
Personnel: 

Past  Project  Leader (s): 
Past  Personnel: 


-  none  - 

-  none  - 

Dr.  Gary  McMurtry,  Dr.  Song  K.  Choi  &  Mr.  Oliver  T.  Easterday 
Mr.  Yann  Douyere,  Mr.  Alan  Parsa  &  Mr.  Max  D.  Cremer 


Objectives 


The  SAUVIM  Mission  Sensor  Package  for  Phase  1  is  designed  to  provide  semi-continuous 
records  of  AUV  water  depth  (pressure),  water  temperature,  conductivity,  computed  salinity, 
dissolved  oxygen,  pH  and  turbidity  for  at  least  eight  hours.  These  parameters  as  well  as  the 
magnetic  signature  of  the  seafloor  can  be  acquired  by  the  SAUVIM  in  survey  mode.  In 
intervention  mode,  the  Mission  Sensor  Package  will  provide  AUV  water  depth  (pressure) 
and  the  water  temperature  and  compositional  parameters  at  a  selected  seafloor  target, 
including  pumped  samples  from  submarine  seeps  or  vents. 


Current  Status  (Tasks  Completed) 


The  task  has  been  completed  in  the  previous  phases.  Refer  to  previous  reports  for  its 
descriptions. 
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Hydrodynamic  Drag  Coefficient  Analysis  (HDCA) 


Project  Leader (s): 
Personnel: 

Past  Project  Leader (s): 
Past  Personnel: 


-  none  - 

-  none  - 

Dr.  Song  K.  Choi,  Dr.  Farzad  Masheyekhi,  Dr.  Junku  Yuh,  Dr. 
Curtis  S.  Ikehara  &  Mr.  Oliver  T.  Easterday 
Mr.  Brian  S.C.  Lau 


Objectives 


Determination  of  the  hydrodynamic  coefficient  via  numerical  solution  of  full  Navier- 
Stokes  equations  using  commercial  CFD  code,  PHOENICS. 

Provide  design  recommendations  for  the  vehicle  fairing  from  the  hydrodynamic  results. 
Perform  experiments  to  verify  and  confirm  the  CFD  results. 


Current  Status  (Tasks  Completed) 


The  task  has  been  completed  in  the  previous  phases.  Refer  to  previous  reports  for  its 
descriptions. 
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Mechanical  Analysis  and  Fabrication  (MAF) 


Project  Leader (s): 
Personnel: 

Past  Project  Leader (s): 
Past  Personnel: 


Dr.  Song  K.  Choi 
-  none  - 

Dr.  Mehrdad  Ghasemi  Nejhad  &  Mr.  Oliver  T.  Easterday 
Dr.  Ali  Yousefpour,  Mr.  Eric  Sung,  Mr.  Bruce  Flegal,  Mr.  Robert 
Ng,  Mr.  Mark  Uyema,  Mr.  Saeid  Pourjalali,  Ms.  Melanie 
Yamauchi  &  Mr.  Reid  Takaiya 


Objectives 


Mechanical  Analysis  and  Fabrication  (MAF)  group  is  responsible  for  designing,  analyzing, 
manufacturing,  and  testing  of  pressure  vessels  and  flooded  fairing  as  well  as  analyzing  the 
metallic  frame  of  the  vehicle. 


Current  Status  (Tasks  Completed) 


The  task  has  been  completed  in  the  previous  phases.  Refer  to  previous  reports  for  its 
descriptions. 
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Mechanical-Electrical  Design  (MED) 


Project  Leader (s): 
Personnel: 


Past  Project  Leader (s): 
Past  Personnel: 


Dr.  Song  K.  Choi,  Dr.  Giacomo  Marani 

Mr.  Kaikala  Rosa,  Mr.  Aaron  Hanai,  Mr.  Christopher  A.  McLeod, 
Mr.  Edgar  Gongora,  Mr.  Scott  Weatherwax,  Mr.  Patrick  Simmons, 
Mr.  Greg  Tamasahi. 

Dr.  Curtis  S.  Ikehara,  Dr.  Junku  Yuh,  Mr.  Gus  Coutsourakis,  Mr. 
Oliver  T.  Easterday  &  Mr.  Michael  E.  West 

Mr.  Ismael  Medrano,  Mr.  Dante  Julian,  Mr.  Stacy  Hanson,  Mr. 
Lawrence  Wong,  Mr.  Mark  Fujita,  Mr.  Dicson  Aggabao,  Mr.  Szu- 
Min  Chang,  Ms.  Colleen  Kaku,  Mr.  Mike  Hall,  Mr.  Tai  Blechta, 
Mr.  Scott  Sufak,  Mr.  Keith  Sunderlin,  Mr.  Clyde  Campos,  Mr. 
Richard  Antunes,  Mr.  John  Lee,  Mr.  Scott  Sufak,  Mr.  Daniel 
Shnidman,  Mr.  Weston  Fujii,  Mr.  John  Lemmond  &  Ms.  Elizabeth 
Shim 


Objectives 


Integrate  mechanical  and  electrical  components  of  the  SAUVIM  vehicle  and  provide  vehicle 
infrastructure  in  terms  of  structure  and  power  to  support  research  aspects  of  SAUVIM  AUV. 
One  of  the  most  relevant  progress  in  the  Phase  III-B  was  the  Thruster  power  system 
upgrade. 


Current  Status 


The  task  has  been  completed  in  the  previous  phases.  Refer  to  previous  reports  for  its 
descriptions. 
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